{ "id": 519303, "title": "Numina 1st Place Solution", "content": "\n

Thank you to AIMO and Kaggle for hosting this very interesting competition - we learned a lot about the mathematical capabilities of open LLMs and also the challenges one faces when trying to run them on limited computational resources. We are also grateful to our fellow competitors who shared various insights during the competition - especially Abdur Rafae, whose public notebook informed a lot of our decisions around model training and inference.

\n

Our solution consisted of three main components:

\n\n

You can find a collection of the models, demo, and validation sets on the Hugging Face Hub here - we will release the training data and code very soon!

\n

We used a mix of open-source libraries to train our models, notably TRL, PyTorch, vLLM, and DeepSpeed. On one node of 8 x H100 GPUs, our models took 10 hours to train.

\n

The training recipe

\n

Links:

\n\n

Our fine-tuning recipe was largely based on the MuMath-Code paper, which involves training the model in two stages:
\n

\n\n

In both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation. To accelerate training, we used the “packing” feature from TRL’s SFTTrainer to concatenate multiple samples in a single chunk of 2048 tokens. All models were trained with gradient checkpointing and sharded with the DeepSpeed ZeRO-3 protocol to ensure the weights, gradients, and optimizer states could fit within the available VRAM. See below for the main hyperparameters we used in each stage:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Stage 1Stage 2
Learning Rate2.00E-052.00E-05
Total Batch Size3232
Block Size20481024
Num Epochs34
LR Schedulercosinecosine
Warmup Ratio0.10.1
\n

Our initial submissions used DeepSeek 7B models that were only fine-tuned on Stage 1, but we found the performance was quite limited, with 8/50 being our best score on the public leaderboard using maj@32. It was Abdur Rafae’s public notebook that prompted us to take a look at integrating code execution in the training recipe and initially, we focused on the Mix of Minimal Optimal Sets (MMOS) dataset as this was described in the title of the notebook. We found that using MMOS improved performance, but was still capped at 16/50 on the public leaderboard with maj@32, likely due to the fact that MMOS only consists of single-turn solutions (i.e. the model only generates a single Python program which is insufficient for hard problems). We later realized that MMOS was a misnomer and that Kaggle notebooks were actually running the DeepSeekMath 7B RL model which is capable of multi-step reasoning and code execution.

\n

It was at this point that we focused our efforts on producing a dataset similar to the one used by the DeepSeekMath Instruct / RL models, and this together with the MuMath-Code recipe led to significant improvements.

\n

We also experimented with fine-tuning on integer-only solutions for Stage 2, but found the model was slightly more robust when training on all solution types (likely due to the limited number of samples left after filtering).

\n

Self-Consistency with Tool-Integrated Reasoning (SC-TIR)

\n

As other competitors noted, this competition posed several challenges with respect to model submission and evaluation:

\n\n

Initially, we used Abdur Rafae’s public notebook for our submissions but found the high variance to be problematic. To handle this, we took a different approach based on extending self-consistency decoding with tool-integrated reasoning (SC-TIR):
\n

\n
    \n
  1. For each problem, copy the input N times to define the initial batch of prompts to feed vLLM. These effectively define the number of candidates one uses for majority voting.
    2. \n
  2. Sample N completions until a complete block of Python code is produced.
    3. \n
  3. Execute each Python block and concatenate the output, including tracebacks if they appear.
    4. \n
  4. Repeat M times to produce a set of reasoning traces with width N and depth M. If a trace fails to produce sensible outputs (e.g. incomplete code blocks) we prune that trace.
    5. \n
  5. Postprocess the solution candidates and then apply majority voting to select the final answer
    6. \n
\n

For our winning submission, we generated N=48 candidates with a depth of M=4. Increasing either parameter did not improve performance and we took a conservative approach to stay within the time limit. In effect, this algorithm lies somewhere in between Self-Consistency with CoT and Tree of Thoughts (shown below), but doesn’t include backtracking or looking ahead to plan the next step of decoding.
\n

\n

We found that SC-TIR improved overall performance and produced significantly less variance on both our internal evaluations and the public leaderboard.

\n

One technical detail worth mentioning is that we found it helpful to quantize the models in 8-bit precision. This was for two reasons:

\n\n

We quantized our models using AutoGPTQ along with the training datasets for calibration. In practice, this led to a small drop in accuracy, but was mostly mitigated by being able to generate many candidates during inference.

\n

Good validation sets are all you need

\n

Links:
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-aime
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-amc
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-4
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-5

\n

To guide model selection, we used four internal validation sets to gauge the performance of our models on math problems of varying difficulty:

\n\n

By using these four validation sets, we were able to pick the most promising models across different training stages and narrow down the choice of hyperparameters. We found that combining small, but representative validation sets with larger ones was useful in this particular competition where each submission is subject to some stochasticity from sampling.

\n

Other ideas we tried

\n

As mentioned above, we tried a few approaches that were ultimately discarded in favor of the MuMath-Code recipe:

\n\n

Another technique we tried was applying Kahneman-Tversky Optimisation (KTO) to new completions sampled from the SFT model. Here the approach was similar to OrcaMath, namely:

\n\n

We found this form of on-policy KTO produced a better model than the SFT one (a few percentage points on our internal evaluations) and scored 27/50 on the public leaderboard. One nice feature of KTO is that you can track the implicit reward during training and this really helps with debugging a run - for example, here’s one of our successful training logs from Weights and Biases where one sees the chosen (i.e. correct) rewards increase over training, while the rejected ones are suppressed.
\n

\n

Unfortunately, we ran out of time to apply this method to our final SFT model, so it is possible we may have solved 1-2 more problems!

\n

We also experimented with applying our SFT recipe to larger models like InternLM-20B, CodeLama-33B, and Mixtral-8x7B, but found that (a) the DeepSeek 7B model is very hard to beat due to their continued pretraining on math, and (b) inference is very slow and we experienced several mysterious timeouts that we couldn’t trace the root cause of.

\n

Another failed experiment includes trying to use reinforcement learning (specifically the REINFORCE-leave-one-out (RLOO) algorithm) with code execution feedback and a discrete reward for correct/incorrect solutions. We applied this to the DeepSeekMath 7B RL model but did not see any significant gains in performance. Given that online methods like RLOO are bottlenecked by text generation and slow to iterate with, we abandoned reinforcement learning in favor of KTO.
\n

\n

On the inference side, we also experimented with:

\n\n

Closing thoughts

\n

We greatly enjoyed participating in this competition and learned a lot from the experience. For future progress prizes, we think it would be possible to increase the performance of open LLMs even further by enabling:

\n", "comments": [ { "id": 2930700, "author_name": "JM", "author_url": "", "post_date": "2024-07-21T10:21:41.297000", "content": "

\"\"

\n

The team has dropped the Datasets: https://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c

\n

And a new SoTA 72B math model wow!!! 75.8% on MATH and scored 34/40 on AIMC 2023 🤯🤯 - insane job to near GPT-4o performance!

\n

@jiali4 @lewtun @hlneevain @edwardbeeching 👏

", "votes": 9, "replies": [ { "id": 2930702, "author_name": "Sparsh Tewatia", "author_url": "", "post_date": "2024-07-21T10:27:24.550000", "content": "

I believe Qwen 2 72b instruct with TIR recipe it is which is not far behind in AMC , but it is in AIME numina shines.

", "votes": 0, "replies": [] }, { "id": 2978405, "author_name": "Guojun1", "author_url": "", "post_date": "2024-09-03T21:10:40.623000", "content": "

I wonder why all models perform consistently worse in the 2024 problems. Is It because the new problems do not appear in the training set?

", "votes": 3, "replies": [] } ] }, { "id": 3021159, "author_name": "Peter CXL", "author_url": "", "post_date": "2024-10-18T08:29:48.650000", "content": "

The hugging face demo link is down ~ https://huggingface.co/spaces/AI-MO/math-olympiad-solver

", "votes": 4, "replies": [] }, { "id": 2916014, "author_name": "Sparsh Tewatia", "author_url": "", "post_date": "2024-07-10T18:09:07.960000", "content": "

Waiting for that Million sample dataset , btw very elegant solution, I think by the next phase of competition we can touch 40-45 if the momentum keeps up.

", "votes": 5, "replies": [ { "id": 2918824, "author_name": "JM", "author_url": "", "post_date": "2024-07-12T14:31:56.637000", "content": "

I think 40-45 is quite optimistic, that’s effectively winning the grand prize, especially given the Kaggle T4 GPU limitation. Maybe if we can fit DeepSeekCoder V2 Lite we are in good shot of 35-40 score IMO.

", "votes": 4, "replies": [] } ] }, { "id": 2915771, "author_name": "JM", "author_url": "", "post_date": "2024-07-10T16:29:50.797000", "content": "

Congratulations to whole team! Those validation sets are going to be key in the next AIMO competition!

\n
\n

8 x H100 GPUs

\n
\n

My poor RTX 4060 ti can not keep up with this 😂

\n

I think adding a nicely tuned PRM for the ToT would take this over >30.. ? Curious if you have the score differences on SC vs ToT.

\n

Also curious on \"we followed Microsoft’s ToRA paper and prompted GPT-4\" - I assume you used GPT-4o ? Willing to let us know your total cost for the data generation?

\n

Can we get the links to the training and KTO data? The git repo looks empty at the moment. Data generation code will be good to see so we can re-run it with tweaks etc.

\n

Everything from the inference code to the data generation and training process easily explains the large gap from 1st to 2nd 👏

", "votes": 5, "replies": [ { "id": 2916292, "author_name": "Jia Li", "author_url": "", "post_date": "2024-07-10T21:35:43.280000", "content": "

Thank you for your message ! Our team is working hard so to release the dataset soon including the KTO one

", "votes": 6, "replies": [] } ] }, { "id": 2917063, "author_name": "ivan", "author_url": "", "post_date": "2024-07-11T13:06:20.260000", "content": "

Congratulations! Thank you for sharing solution, but I think you hide the most interesting part - how you jump from 16/50 to 27/50? It reminds me the \"How to draw an owl\" meme :) Could you clarify this please? How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further? etc.

", "votes": 6, "replies": [ { "id": 2917230, "author_name": "Jia Li", "author_url": "", "post_date": "2024-07-11T14:55:30.417000", "content": "

how you jump from 16/50 to 27/50?
\nMaybe it is not very clear in the post, but we manage to get very good performance when we switch from MMOS to TORA

\n

How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further?
\nWe are working hard on a dataset technical report and are releasing the report and the dataset next week. We build the dataset first, then tried to leverage the dataset for the competition. It has around 1M samples as mentioned previously. We believe there are still a lot of potential of this dataset, before scaling it to a next level.

", "votes": 5, "replies": [ { "id": 2925613, "author_name": "yusu li", "author_url": "", "post_date": "2024-07-17T09:53:06.573000", "content": "

Thank you very much for sharing this information. I appreciate your effort in working on the dataset technical report and planning to release it next week. Could you please let me know when exactly the report will be made public? I'm eager to learn more about your dataset and its potential.

", "votes": 0, "replies": [] }, { "id": 2930688, "author_name": "Jia Li", "author_url": "", "post_date": "2024-07-21T09:59:43.590000", "content": "

Here is the release of the datasets and the report:
\nhttps://github.com/project-numina/aimo-progress-prize
\nhttps://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c

", "votes": 6, "replies": [] }, { "id": 2966255, "author_name": "KKY", "author_url": "", "post_date": "2024-08-21T16:55:53.723000", "content": "

Thanks for sharing the code and dataset! Awesome solution

", "votes": 0, "replies": [] } ] } ] }, { "id": 2915893, "author_name": "Ali", "author_url": "", "post_date": "2024-07-10T17:30:46.810000", "content": "

Impressive work, Congratulations.
\n8 x H100 GPUs are waaaaay beyond my budget 😅

\n

I have a quick question, your fine-tuning did it include AIME Problem Set: 1983-2024?

", "votes": 6, "replies": [ { "id": 2916299, "author_name": "Jia Li", "author_url": "", "post_date": "2024-07-10T21:41:37.673000", "content": "

You are right ! We have included this data in our fine-tuning. More specifically, AMIE 2022-2024 have been kept as validation set https://huggingface.co/datasets/AI-MO/aimo-validation-aime and the others are in the training set. We will provide more details on the dataset in the next coming days

", "votes": 3, "replies": [] } ] }, { "id": 2934561, "author_name": "Commit2Cosmos", "author_url": "", "post_date": "2024-07-24T14:46:08.417000", "content": "

Great work! Thank you for detailed explanations and amazing validation sets! Looking forward to your solutions for further AIMO-related competitions!

", "votes": 1, "replies": [] }, { "id": 2916792, "author_name": "deeeeeeeplearning", "author_url": "", "post_date": "2024-07-11T08:57:31.037000", "content": "

Congratulations! Could you show part of the datasets used for training in the stage 1 and 2? I am particularly interested in the notes on how the dataset for the stage 2 was collected.

", "votes": 1, "replies": [ { "id": 2917243, "author_name": "Jia Li", "author_url": "", "post_date": "2024-07-11T14:59:41.007000", "content": "

Hi ! We are releasing a report and the dataset next week. You should be able to find answer to your question and play with it !

", "votes": 1, "replies": [] } ] }, { "id": 2916280, "author_name": "opamusora (Ivan Viakhirev)", "author_url": "", "post_date": "2024-07-10T21:16:17.680000", "content": "

Congratulations!!! new SOTA in math? 😅

\n

really great solution, tree of thoughts part and learning python representations of math problems is really smart and interesting solution

", "votes": 1, "replies": [] }, { "id": 2916024, "author_name": "Omar Almusa", "author_url": "", "post_date": "2024-07-10T18:14:46.663000", "content": "
\n

8 x H100 GPUs

\n
\n

no way….

", "votes": 1, "replies": [ { "id": 2916287, "author_name": "Sparsh Tewatia", "author_url": "", "post_date": "2024-07-10T21:22:00.980000", "content": "

bro people from hugginface , mistral ai , what do you expect ??

", "votes": 5, "replies": [ { "id": 2916545, "author_name": "Omar Almusa", "author_url": "", "post_date": "2024-07-11T04:54:15.713000", "content": "

Wait i didn't know that…

", "votes": 1, "replies": [] } ] } ] }, { "id": 3426769, "author_name": "Biren kumar Nayak", "author_url": "", "post_date": "2026-03-23T07:36:49.283000", "content": "

i understood

", "votes": 0, "replies": [] }, { "id": 3197258, "author_name": "hato bai", "author_url": "", "post_date": "2025-05-08T01:18:52.557000", "content": "

Where there is a will, there is a way.

", "votes": 0, "replies": [] }, { "id": 3088269, "author_name": "Sangram Patil", "author_url": "", "post_date": "2025-01-04T13:41:08.030000", "content": "

why you chose fine-tune DeepSeekMath-Base 7B there is any specific reason, because you have 2 or more option rho,  qwen ect? 

", "votes": 0, "replies": [] }, { "id": 3037274, "author_name": "Dalton Omondi", "author_url": "", "post_date": "2024-11-05T14:53:07.980000", "content": "

I loved this notebook. It is very informative especially for the Progress Prize 2. Thank you so much project Numina

", "votes": 0, "replies": [] }, { "id": 2932742, "author_name": "Datamus", "author_url": "", "post_date": "2024-07-23T07:55:03.533000", "content": "

Congrats on the 1st place. Insightful read, thanks for sharing!

", "votes": 0, "replies": [] }, { "id": 2926482, "author_name": "Tudor Stoica", "author_url": "", "post_date": "2024-07-17T18:36:52.957000", "content": "

Congrats! Super impressive :)

", "votes": 0, "replies": [] }, { "id": 2924765, "author_name": "Ziad Eltantawe", "author_url": "", "post_date": "2024-07-16T17:53:05.097000", "content": "

Impressive work

", "votes": 0, "replies": [] }, { "id": 2924393, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2024-07-16T13:43:46.570000", "content": "

Thanks for the solution description, could you elaborate more on why you were not confident on LoRA?

\n
\n

In both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation.

\n
", "votes": 0, "replies": [ { "id": 2925021, "author_name": "Lewis Tunstall", "author_url": "", "post_date": "2024-07-16T21:22:07.673000", "content": "

It's mostly informed by our experience training chat models like Zephyr where we saw LoRA can match the performance of full training, but at the expense of adding new hyperparameters to the mix (rank & alpha) which require more time to get right. We think DoRA (or QDoRA) should work better, but we have not tinkered with it yet and were already tight for time. This blog post from answer.ai has some nice details on what it takes to make QDoRA work: http://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html

", "votes": 7, "replies": [ { "id": 2925402, "author_name": "Guillermo Barbadillo", "author_url": "", "post_date": "2024-07-17T06:51:28.130000", "content": "

Thanks for the reply and the link, it's a very nice read.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2921414, "author_name": "corochann", "author_url": "", "post_date": "2024-07-14T11:18:59.690000", "content": "

Congratulations and thaks for sharing impressive solution. It was very interesting.

\n

I think code execution feedback will be the key trend for better quality synthetic data generation, especially effective for math and coding domains.

", "votes": 0, "replies": [] }, { "id": 2918149, "author_name": "C R Suthikshn Kumar", "author_url": "", "post_date": "2024-07-12T03:54:46.117000", "content": "

Congratulations on winning this competition. Thanks for sharing the details of your solution.

", "votes": 0, "replies": [] }, { "id": 3078837, "author_name": "", "author_url": "", "post_date": "2024-12-22T22:51:51.157000", "content": "", "votes": 0, "replies": [] }, { "id": 2915681, "author_name": "", "author_url": "", "post_date": "2024-07-10T15:53:12.393000", "content": "", "votes": 4, "replies": [] } ], "comment_count": 36, "messages": [ { "id": 2915580, "postDate": "2024-07-10T15:14:59.983Z", "content": "\n

Thank you to AIMO and Kaggle for hosting this very interesting competition - we learned a lot about the mathematical capabilities of open LLMs and also the challenges one faces when trying to run them on limited computational resources. We are also grateful to our fellow competitors who shared various insights during the competition - especially Abdur Rafae, whose public notebook informed a lot of our decisions around model training and inference.

\n

Our solution consisted of three main components:

\n\n

You can find a collection of the models, demo, and validation sets on the Hugging Face Hub here - we will release the training data and code very soon!

\n

We used a mix of open-source libraries to train our models, notably TRL, PyTorch, vLLM, and DeepSpeed. On one node of 8 x H100 GPUs, our models took 10 hours to train.

\n

The training recipe

\n

Links:

\n\n

Our fine-tuning recipe was largely based on the MuMath-Code paper, which involves training the model in two stages:
\n

\n\n

In both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation. To accelerate training, we used the “packing” feature from TRL’s SFTTrainer to concatenate multiple samples in a single chunk of 2048 tokens. All models were trained with gradient checkpointing and sharded with the DeepSpeed ZeRO-3 protocol to ensure the weights, gradients, and optimizer states could fit within the available VRAM. See below for the main hyperparameters we used in each stage:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
Stage 1Stage 2
Learning Rate2.00E-052.00E-05
Total Batch Size3232
Block Size20481024
Num Epochs34
LR Schedulercosinecosine
Warmup Ratio0.10.1
\n

Our initial submissions used DeepSeek 7B models that were only fine-tuned on Stage 1, but we found the performance was quite limited, with 8/50 being our best score on the public leaderboard using maj@32. It was Abdur Rafae’s public notebook that prompted us to take a look at integrating code execution in the training recipe and initially, we focused on the Mix of Minimal Optimal Sets (MMOS) dataset as this was described in the title of the notebook. We found that using MMOS improved performance, but was still capped at 16/50 on the public leaderboard with maj@32, likely due to the fact that MMOS only consists of single-turn solutions (i.e. the model only generates a single Python program which is insufficient for hard problems). We later realized that MMOS was a misnomer and that Kaggle notebooks were actually running the DeepSeekMath 7B RL model which is capable of multi-step reasoning and code execution.

\n

It was at this point that we focused our efforts on producing a dataset similar to the one used by the DeepSeekMath Instruct / RL models, and this together with the MuMath-Code recipe led to significant improvements.

\n

We also experimented with fine-tuning on integer-only solutions for Stage 2, but found the model was slightly more robust when training on all solution types (likely due to the limited number of samples left after filtering).

\n

Self-Consistency with Tool-Integrated Reasoning (SC-TIR)

\n

As other competitors noted, this competition posed several challenges with respect to model submission and evaluation:

\n\n

Initially, we used Abdur Rafae’s public notebook for our submissions but found the high variance to be problematic. To handle this, we took a different approach based on extending self-consistency decoding with tool-integrated reasoning (SC-TIR):
\n

\n
    \n
  1. For each problem, copy the input N times to define the initial batch of prompts to feed vLLM. These effectively define the number of candidates one uses for majority voting.
    2. \n
  2. Sample N completions until a complete block of Python code is produced.
    3. \n
  3. Execute each Python block and concatenate the output, including tracebacks if they appear.
    4. \n
  4. Repeat M times to produce a set of reasoning traces with width N and depth M. If a trace fails to produce sensible outputs (e.g. incomplete code blocks) we prune that trace.
    5. \n
  5. Postprocess the solution candidates and then apply majority voting to select the final answer
    6. \n
\n

For our winning submission, we generated N=48 candidates with a depth of M=4. Increasing either parameter did not improve performance and we took a conservative approach to stay within the time limit. In effect, this algorithm lies somewhere in between Self-Consistency with CoT and Tree of Thoughts (shown below), but doesn’t include backtracking or looking ahead to plan the next step of decoding.
\n

\n

We found that SC-TIR improved overall performance and produced significantly less variance on both our internal evaluations and the public leaderboard.

\n

One technical detail worth mentioning is that we found it helpful to quantize the models in 8-bit precision. This was for two reasons:

\n\n

We quantized our models using AutoGPTQ along with the training datasets for calibration. In practice, this led to a small drop in accuracy, but was mostly mitigated by being able to generate many candidates during inference.

\n

Good validation sets are all you need

\n

Links:
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-aime
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-amc
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-4
\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-5

\n

To guide model selection, we used four internal validation sets to gauge the performance of our models on math problems of varying difficulty:

\n\n

By using these four validation sets, we were able to pick the most promising models across different training stages and narrow down the choice of hyperparameters. We found that combining small, but representative validation sets with larger ones was useful in this particular competition where each submission is subject to some stochasticity from sampling.

\n

Other ideas we tried

\n

As mentioned above, we tried a few approaches that were ultimately discarded in favor of the MuMath-Code recipe:

\n\n

Another technique we tried was applying Kahneman-Tversky Optimisation (KTO) to new completions sampled from the SFT model. Here the approach was similar to OrcaMath, namely:

\n\n

We found this form of on-policy KTO produced a better model than the SFT one (a few percentage points on our internal evaluations) and scored 27/50 on the public leaderboard. One nice feature of KTO is that you can track the implicit reward during training and this really helps with debugging a run - for example, here’s one of our successful training logs from Weights and Biases where one sees the chosen (i.e. correct) rewards increase over training, while the rejected ones are suppressed.
\n

\n

Unfortunately, we ran out of time to apply this method to our final SFT model, so it is possible we may have solved 1-2 more problems!

\n

We also experimented with applying our SFT recipe to larger models like InternLM-20B, CodeLama-33B, and Mixtral-8x7B, but found that (a) the DeepSeek 7B model is very hard to beat due to their continued pretraining on math, and (b) inference is very slow and we experienced several mysterious timeouts that we couldn’t trace the root cause of.

\n

Another failed experiment includes trying to use reinforcement learning (specifically the REINFORCE-leave-one-out (RLOO) algorithm) with code execution feedback and a discrete reward for correct/incorrect solutions. We applied this to the DeepSeekMath 7B RL model but did not see any significant gains in performance. Given that online methods like RLOO are bottlenecked by text generation and slow to iterate with, we abandoned reinforcement learning in favor of KTO.
\n

\n

On the inference side, we also experimented with:

\n\n

Closing thoughts

\n

We greatly enjoyed participating in this competition and learned a lot from the experience. For future progress prizes, we think it would be possible to increase the performance of open LLMs even further by enabling:

\n", "rawMarkdown": "- Notebook: https://www.kaggle.com/code/lewtun/numina-1st-place-solution\n- Model: https://huggingface.co/AI-MO/NuminaMath-7B-TIR\n- Demo: https://huggingface.co/spaces/AI-MO/math-olympiad-solver\n\nThank you to AIMO and Kaggle for hosting this very interesting competition - we learned a lot about the mathematical capabilities of open LLMs and also the challenges one faces when trying to run them on limited computational resources. We are also grateful to our fellow competitors who shared various insights during the competition - especially [Abdur Rafae](https://www.kaggle.com/abdurrafae), whose [public notebook](https://www.kaggle.com/code/abdurrafae/improved-code-interpretation) informed a lot of our decisions around model training and inference.\n\nOur solution consisted of three main components: \n\n- a recipe to fine-tune [DeepSeekMath-Base 7B](https://huggingface.co/deepseek-ai/deepseek-math-7b-base) to act as a “reasoning agent” that can solve mathematical problems via a mix of natural language and the use of the Python REPL.\n- a decoding algorithm for tool-integrated reasoning (TIR) with code execution feedback to generate solution candidates during inference. \n- a variety of internal validation sets that we used to guide model selection and avoid overfitting to the public leaderboard.\n\nYou can find a collection of the models, demo, and validation sets on the Hugging Face Hub here - we will release the training data and code very soon!\n\nWe used a mix of open-source libraries to train our models, notably TRL, PyTorch, vLLM, and DeepSpeed. On one node of 8 x H100 GPUs, our models took 10 hours to train.\n\n# The training recipe\n\nLinks:\n- Our final model: [AI-MO/NuminaMath-7B-TIR](https://huggingface.co/AI-MO/NuminaMath-7B-TIR)\n- Our model for Kaggle in quantized 8-bit precision: [AI-MO/NuminaMath-7B-TIR-GPTQ](https://huggingface.co/AI-MO/NuminaMath-7B-TIR-GPTQ)\n\nOur fine-tuning recipe was largely based on the [MuMath-Code paper](https://arxiv.org/abs/2405.07551), which involves training the model in two stages:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2F16c7ceec1b0a85897e12a479c7346960%2Fmumath.png?generation=1720623286883125&alt=media)\n\n- **Stage 1**: fine-tune the base model on a large, diverse dataset of natural language math problems and solutions, where each solution is templated with Chain of Thought (CoT) to facilitate learning. \n- **Stage 2**: fine-tune the model from Stage 1 on a synthetic dataset of tool-integrated reasoning, where each math problem is decomposed into a sequence of rationales, Python programs, and their outputs. Here we followed Microsoft’s [ToRA](https://arxiv.org/abs/2309.17452) paper and prompted GPT-4 to produce solutions in the ToRA format with code execution feedback. Fine-tuning on this data produces a reasoning agent that can solve mathematical problems via a mix of natural language reasoning and use of the Python REPL to compute intermediate results (see screenshot below).\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2F7e8fe66053516013d69a787a85f15e70%2Ftora.png?generation=1720623321486212&alt=media)\n\nIn both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation. To accelerate training, we used the “packing” feature from TRL’s SFTTrainer to concatenate multiple samples in a single chunk of 2048 tokens. All models were trained with gradient checkpointing and sharded with the DeepSpeed ZeRO-3 protocol to ensure the weights, gradients, and optimizer states could fit within the available VRAM. See below for the main hyperparameters we used in each stage:\n\n| | Stage 1 | Stage 2 |\n|-----------------|-------------|-------------|\n| **Learning Rate** | 2.00E-05 | 2.00E-05 |\n| **Total Batch Size**| 32 | 32 |\n| **Block Size** | 2048 | 1024 |\n| **Num Epochs** | 3 | 4 |\n| **LR Scheduler** | cosine | cosine |\n| **Warmup Ratio** | 0.1 | 0.1 |\n\nOur initial submissions used DeepSeek 7B models that were only fine-tuned on Stage 1, but we found the performance was quite limited, with 8/50 being our best score on the public leaderboard using maj@32. It was Abdur Rafae’s public notebook that prompted us to take a look at integrating code execution in the training recipe and initially, we focused on the [Mix of Minimal Optimal Sets (MMOS)](https://github.com/cyzhh/MMOS) dataset as this was described in the title of the notebook. We found that using MMOS improved performance, but was still capped at 16/50 on the public leaderboard with maj@32, likely due to the fact that MMOS only consists of single-turn solutions (i.e. the model only generates a single Python program which is insufficient for hard problems). We later realized that MMOS was a misnomer and that Kaggle notebooks were actually running the DeepSeekMath 7B RL model which is capable of multi-step reasoning and code execution. \n\nIt was at this point that we focused our efforts on producing a dataset similar to the one used by the DeepSeekMath Instruct / RL models, and this together with the MuMath-Code recipe led to significant improvements.\n\nWe also experimented with fine-tuning on integer-only solutions for Stage 2, but found the model was slightly more robust when training on all solution types (likely due to the limited number of samples left after filtering).\n\n# Self-Consistency with Tool-Integrated Reasoning (SC-TIR)\nAs other competitors noted, this competition posed several challenges with respect to model submission and evaluation:\n\n- The evaluation API provides problems in random order, so tactics like early stopping produce high variance because one run may have more hard problems at the start which leaves less time for the remainder (and vice versa)\n- Most innovation in LLM inference requires access to modern GPUs, so standard methods like Flash Attention 2 or torch.compile do not work on the T4 GPUs. Similarly, modern data types like bfloat16 are not supported which prompted us to explore post-training quantization methods like AWQ and GPTQ.\n\nInitially, we used Abdur Rafae’s public notebook for our submissions but found the high variance to be problematic. To handle this, we took a different approach based on extending self-consistency decoding with tool-integrated reasoning (SC-TIR): \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2F782cadb630e503b5f35c237fd651faaf%2Fsc-tir.png?generation=1720623396756900&alt=media)\n\n1. For each problem, copy the input N times to define the initial batch of prompts to feed vLLM. These effectively define the number of candidates one uses for majority voting.\n2. Sample N completions until a complete block of Python code is produced.\n3. Execute each Python block and concatenate the output, including tracebacks if they appear.\n4. Repeat M times to produce a set of reasoning traces with width N and depth M. If a trace fails to produce sensible outputs (e.g. incomplete code blocks) we prune that trace.\n5. Postprocess the solution candidates and then apply majority voting to select the final answer\n\nFor our winning submission, we generated N=48 candidates with a depth of M=4. Increasing either parameter did not improve performance and we took a conservative approach to stay within the time limit. In effect, this algorithm lies somewhere in between Self-Consistency with CoT and Tree of Thoughts (shown below), but doesn’t include backtracking or looking ahead to plan the next step of decoding.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2F0f393f7fb7224bd26b1e7a421f73c195%2Ftot.png?generation=1720623414056019&alt=media)\n\nWe found that SC-TIR improved overall performance and produced significantly less variance on both our internal evaluations and the public leaderboard. \n\nOne technical detail worth mentioning is that we found it helpful to quantize the models in 8-bit precision. This was for two reasons:\n\n- It is very slow to upload models to the Kaggle Hub and compressing the model made this step twice as fast.\n- A 16-bit model consumes approximately 32GB VRAM just to load the weights. With 2xT4s, this would have required manipulating the KV cache to run fast and we found it beneficial to tradeoff model precision for speed.\n\nWe quantized our models using [AutoGPTQ](https://github.com/AutoGPTQ/AutoGPTQ) along with the training datasets for calibration. In practice, this led to a small drop in accuracy, but was mostly mitigated by being able to generate many candidates during inference.\n\n# Good validation sets are all you need\nLinks:\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-aime\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-amc\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-4\nhttps://huggingface.co/datasets/AI-MO/aimo-validation-math-level-5\n\nTo guide model selection, we used four internal validation sets to gauge the performance of our models on math problems of varying difficulty:\n- AMC: we picked all the problems from years 2022 and 2023 of AMC12 and kept those that can be converted to integer outputs. This results in a dataset of 83 problems. This validation set was designed to mimic the private test set on Kaggle and we found our models could solve about 60-65% of these problems. To measure the variance we ran each evaluation with 5-10 different seeds and typically saw variations of around 1-3% with our TIR algorithm.\n- AIME: we picked all the problems from AIME 22, AIME 23, and AIME 24 to measure how well our models could perform on difficult problems, as well as to gauge the most common failure modes. As above, we ran each evaluation with 5-10 seeds to measure variation.\n- MATH levels 4 & 5: we were worried the Medium/Hard validation sets were too small to give a reliable signal, so we also filtered the level 4 and 5 samples from the MATH test set for those with integer solutions. This gave about 750 problems per level to evaluate on and we did so with a single seed per model.\n\nBy using these four validation sets, we were able to pick the most promising models across different training stages and narrow down the choice of hyperparameters. We found that combining small, but representative validation sets with larger ones was useful in this particular competition where each submission is subject to some stochasticity from sampling.\n\n# Other ideas we tried\nAs mentioned above, we tried a few approaches that were ultimately discarded in favor of the MuMath-Code recipe:\n\n- Training a pure CoT model and using majority voting for evaluation\n- Training an MMOS model to solve problems with Python in a single step\n\nAnother technique we tried was applying [Kahneman-Tversky Optimisation (KTO)](https://arxiv.org/abs/2402.01306) to new completions sampled from the SFT model. Here the approach was similar to OrcaMath, namely:\n\n- Sample 4 completions per problem with the SFT model, using interleaved rationales and code execution. We used the SFT dataset from Stage 2 as the source of prompts.\n- Extract the answer and compare it with the ground truth. If correct, label the sample as positive, else negative.\n- Apply KTO to the SFT model on this dataset\n\nWe found this form of on-policy KTO produced a better model than the SFT one (a few percentage points on our internal evaluations) and scored 27/50 on the public leaderboard. One nice feature of KTO is that you can track the implicit reward during training and this really helps with debugging a run - for example, here’s one of our successful training logs from Weights and Biases where one sees the chosen (i.e. correct) rewards increase over training, while the rejected ones are suppressed.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2F968c335a19316a3555fc85926dcaf45d%2Fkto-curves.png?generation=1720623447295151&alt=media)\n\nUnfortunately, we ran out of time to apply this method to our final SFT model, so it is possible we may have solved 1-2 more problems!\n\nWe also experimented with applying our SFT recipe to larger models like InternLM-20B, CodeLama-33B, and Mixtral-8x7B, but found that (a) the DeepSeek 7B model is very hard to beat due to their continued pretraining on math, and (b) inference is very slow and we experienced several mysterious timeouts that we couldn’t trace the root cause of. \n\nAnother failed experiment includes trying to use reinforcement learning (specifically the [REINFORCE-leave-one-out (RLOO) algorithm](https://arxiv.org/abs/2402.14740)) with code execution feedback and a discrete reward for correct/incorrect solutions. We applied this to the DeepSeekMath 7B RL model but did not see any significant gains in performance. Given that online methods like RLOO are bottlenecked by text generation and slow to iterate with, we abandoned reinforcement learning in favor of KTO.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F264297%2Fdefc73c0664ae3f88c3d530d5f7732e2%2Frloo-curves_720.png?generation=1720623500018321&alt=media)\n\nOn the inference side, we also experimented with:\n\n- Using a static KV cache and torch compilation. We found we were able to speed up generation in native transformers code by 2-3x on a H100, but hit a variety of cryptic errors on the Kaggle T4s, mostly due to the lack of support for model sharding with torch compilation in Hugging Face accelerate.\n- A variety of model merging techniques like DARE, TIES, and WARP. Here we used mergekit (https://github.com/arcee-ai/mergekit) to merge the SFT and KTO models, or the SFT models with the public DeepSeekMath ones. Overall we found these merges led to either significant regressions on our internal evaluations or we ran out of time to explore this more deeply.\n\n# Closing thoughts\nWe greatly enjoyed participating in this competition and learned a lot from the experience. For future progress prizes, we think it would be possible to increase the performance of open LLMs even further by enabling:\n\n- Evaluation on modern GPUs like A100s or H100s. As noted earlier, much of the recent progress in LLM inference optimization relies on the newer compute architectures and data types, and using GPUs with more VRAM would enable people to experiment with more powerful models.\n- Relaxing the pretrained model cutoff date to the start date of the competition. With the current pace of LLM development, this would enable competitors to use the most powerful open LLM available and provide a better measure of how far we are compared to human baselines or proprietary models.\n\n", "votes": 191 }, { "id": 2930700, "postDate": "2024-07-21T10:21:41.297Z", "content": "

\"\"

\n

The team has dropped the Datasets: https://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c

\n

And a new SoTA 72B math model wow!!! 75.8% on MATH and scored 34/40 on AIMC 2023 🤯🤯 - insane job to near GPT-4o performance!

\n

@jiali4 @lewtun @hlneevain @edwardbeeching 👏

", "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F470189%2Fb8e2662c302eb75333bf6f744b3902d8%2Fnumina.jfif?generation=1721557051176257&alt=media)\n\nThe team has dropped the Datasets: https://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c\n\nAnd a new SoTA 72B math model wow!!! 75.8% on MATH and scored 34/40 on AIMC 2023 🤯🤯 - insane job to near GPT-4o performance!\n\n@jiali4 @lewtun @hlneevain @edwardbeeching 👏", "votes": 9, "replies": [ { "id": 2930702, "postDate": "2024-07-21T10:27:24.550Z", "content": "

I believe Qwen 2 72b instruct with TIR recipe it is which is not far behind in AMC , but it is in AIME numina shines.

", "rawMarkdown": "I believe Qwen 2 72b instruct with TIR recipe it is which is not far behind in AMC , but it is in AIME numina shines." }, { "id": 2978405, "postDate": "2024-09-03T21:10:40.623Z", "content": "

I wonder why all models perform consistently worse in the 2024 problems. Is It because the new problems do not appear in the training set?

", "rawMarkdown": "I wonder why all models perform consistently worse in the 2024 problems. Is It because the new problems do not appear in the training set? ", "votes": 3 } ] }, { "id": 3021159, "postDate": "2024-10-18T08:29:48.650Z", "content": "

The hugging face demo link is down ~ https://huggingface.co/spaces/AI-MO/math-olympiad-solver

", "rawMarkdown": "The hugging face demo link is down ~ https://huggingface.co/spaces/AI-MO/math-olympiad-solver", "votes": 4 }, { "id": 2916014, "postDate": "2024-07-10T18:09:07.960Z", "content": "

Waiting for that Million sample dataset , btw very elegant solution, I think by the next phase of competition we can touch 40-45 if the momentum keeps up.

", "rawMarkdown": "Waiting for that Million sample dataset , btw very elegant solution, I think by the next phase of competition we can touch 40-45 if the momentum keeps up.", "votes": 5, "replies": [ { "id": 2918824, "postDate": "2024-07-12T14:31:56.637Z", "content": "

I think 40-45 is quite optimistic, that’s effectively winning the grand prize, especially given the Kaggle T4 GPU limitation. Maybe if we can fit DeepSeekCoder V2 Lite we are in good shot of 35-40 score IMO.

", "rawMarkdown": "I think 40-45 is quite optimistic, that’s effectively winning the grand prize, especially given the Kaggle T4 GPU limitation. Maybe if we can fit DeepSeekCoder V2 Lite we are in good shot of 35-40 score IMO.", "votes": 4 } ] }, { "id": 2915771, "postDate": "2024-07-10T16:29:50.797Z", "content": "

Congratulations to whole team! Those validation sets are going to be key in the next AIMO competition!

\n
\n

8 x H100 GPUs

\n
\n

My poor RTX 4060 ti can not keep up with this 😂

\n

I think adding a nicely tuned PRM for the ToT would take this over >30.. ? Curious if you have the score differences on SC vs ToT.

\n

Also curious on \"we followed Microsoft’s ToRA paper and prompted GPT-4\" - I assume you used GPT-4o ? Willing to let us know your total cost for the data generation?

\n

Can we get the links to the training and KTO data? The git repo looks empty at the moment. Data generation code will be good to see so we can re-run it with tweaks etc.

\n

Everything from the inference code to the data generation and training process easily explains the large gap from 1st to 2nd 👏

", "rawMarkdown": "Congratulations to whole team! Those validation sets are going to be key in the next AIMO competition!\n\n>8 x H100 GPUs\n\nMy poor RTX 4060 ti can not keep up with this 😂\n\nI think adding a nicely tuned PRM for the ToT would take this over >30.. ? Curious if you have the score differences on SC vs ToT.\n\nAlso curious on \"**we followed Microsoft’s ToRA paper and prompted GPT-4**\" - I assume you used GPT-4o ? Willing to let us know your total cost for the data generation? \n\nCan we get the links to the training and KTO data? The git repo looks empty at the moment. Data generation code will be good to see so we can re-run it with tweaks etc.\n\nEverything from the inference code to the data generation and training process easily explains the large gap from 1st to 2nd 👏", "votes": 5, "replies": [ { "id": 2916292, "postDate": "2024-07-10T21:35:43.280Z", "content": "

Thank you for your message ! Our team is working hard so to release the dataset soon including the KTO one

", "rawMarkdown": "Thank you for your message ! Our team is working hard so to release the dataset soon including the KTO one", "votes": 6 } ] }, { "id": 2917063, "postDate": "2024-07-11T13:06:20.260Z", "content": "

Congratulations! Thank you for sharing solution, but I think you hide the most interesting part - how you jump from 16/50 to 27/50? It reminds me the \"How to draw an owl\" meme :) Could you clarify this please? How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further? etc.

", "rawMarkdown": "Congratulations! Thank you for sharing solution, but I think you hide the most interesting part - how you jump from 16/50 to 27/50? It reminds me the \"How to draw an owl\" meme :) Could you clarify this please? How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further? etc.", "votes": 6, "replies": [ { "id": 2917230, "postDate": "2024-07-11T14:55:30.417Z", "content": "

how you jump from 16/50 to 27/50?
\nMaybe it is not very clear in the post, but we manage to get very good performance when we switch from MMOS to TORA

\n

How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further?
\nWe are working hard on a dataset technical report and are releasing the report and the dataset next week. We build the dataset first, then tried to leverage the dataset for the competition. It has around 1M samples as mentioned previously. We believe there are still a lot of potential of this dataset, before scaling it to a next level.

", "rawMarkdown": "*how you jump from 16/50 to 27/50?*\nMaybe it is not very clear in the post, but we manage to get very good performance when we switch from MMOS to TORA\n\n*How much new data are you using at each stage? How long does it take to generate this data? How did you do this? How does this affect the LB and your local AMC/AIME/MATH scores? Did you add data iteratively? If you add more data, will it improve further?*\nWe are working hard on a dataset technical report and are releasing the report and the dataset next week. We build the dataset first, then tried to leverage the dataset for the competition. It has around 1M samples as mentioned previously. We believe there are still a lot of potential of this dataset, before scaling it to a next level.", "votes": 5, "replies": [ { "id": 2925613, "postDate": "2024-07-17T09:53:06.573Z", "content": "

Thank you very much for sharing this information. I appreciate your effort in working on the dataset technical report and planning to release it next week. Could you please let me know when exactly the report will be made public? I'm eager to learn more about your dataset and its potential.

", "rawMarkdown": "Thank you very much for sharing this information. I appreciate your effort in working on the dataset technical report and planning to release it next week. Could you please let me know when exactly the report will be made public? I'm eager to learn more about your dataset and its potential." }, { "id": 2930688, "postDate": "2024-07-21T09:59:43.590Z", "content": "

Here is the release of the datasets and the report:
\nhttps://github.com/project-numina/aimo-progress-prize
\nhttps://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c

", "rawMarkdown": "Here is the release of the datasets and the report:\nhttps://github.com/project-numina/aimo-progress-prize\nhttps://huggingface.co/collections/AI-MO/numinamath-6697df380293bcfdbc1d978c", "votes": 6 }, { "id": 2966255, "postDate": "2024-08-21T16:55:53.723Z", "content": "

Thanks for sharing the code and dataset! Awesome solution

", "rawMarkdown": "Thanks for sharing the code and dataset! Awesome solution" } ] } ] }, { "id": 2915893, "postDate": "2024-07-10T17:30:46.810Z", "content": "

Impressive work, Congratulations.
\n8 x H100 GPUs are waaaaay beyond my budget 😅

\n

I have a quick question, your fine-tuning did it include AIME Problem Set: 1983-2024?

", "rawMarkdown": "Impressive work, Congratulations. \n8 x H100 GPUs are waaaaay beyond my budget 😅\n\nI have a quick question, your fine-tuning did it include AIME Problem Set: 1983-2024? ", "votes": 6, "replies": [ { "id": 2916299, "postDate": "2024-07-10T21:41:37.673Z", "content": "

You are right ! We have included this data in our fine-tuning. More specifically, AMIE 2022-2024 have been kept as validation set https://huggingface.co/datasets/AI-MO/aimo-validation-aime and the others are in the training set. We will provide more details on the dataset in the next coming days

", "rawMarkdown": "You are right ! We have included this data in our fine-tuning. More specifically, AMIE 2022-2024 have been kept as validation set https://huggingface.co/datasets/AI-MO/aimo-validation-aime and the others are in the training set. We will provide more details on the dataset in the next coming days ", "votes": 3 } ] }, { "id": 2934561, "postDate": "2024-07-24T14:46:08.417Z", "content": "

Great work! Thank you for detailed explanations and amazing validation sets! Looking forward to your solutions for further AIMO-related competitions!

", "rawMarkdown": "Great work! Thank you for detailed explanations and amazing validation sets! Looking forward to your solutions for further AIMO-related competitions!", "votes": 1 }, { "id": 2916792, "postDate": "2024-07-11T08:57:31.037Z", "content": "

Congratulations! Could you show part of the datasets used for training in the stage 1 and 2? I am particularly interested in the notes on how the dataset for the stage 2 was collected.

", "rawMarkdown": "Congratulations! Could you show part of the datasets used for training in the stage 1 and 2? I am particularly interested in the notes on how the dataset for the stage 2 was collected. ", "votes": 1, "replies": [ { "id": 2917243, "postDate": "2024-07-11T14:59:41.007Z", "content": "

Hi ! We are releasing a report and the dataset next week. You should be able to find answer to your question and play with it !

", "rawMarkdown": "Hi ! We are releasing a report and the dataset next week. You should be able to find answer to your question and play with it !", "votes": 1 } ] }, { "id": 2916280, "postDate": "2024-07-10T21:16:17.680Z", "content": "

Congratulations!!! new SOTA in math? 😅

\n

really great solution, tree of thoughts part and learning python representations of math problems is really smart and interesting solution

", "rawMarkdown": "Congratulations!!! new SOTA in math? 😅\n\nreally great solution, tree of thoughts part and learning python representations of math problems is really smart and interesting solution", "votes": 1 }, { "id": 2916024, "postDate": "2024-07-10T18:14:46.663Z", "content": "
\n

8 x H100 GPUs

\n
\n

no way….

", "rawMarkdown": ">8 x H100 GPUs\n\nno way....", "votes": 1, "replies": [ { "id": 2916287, "postDate": "2024-07-10T21:22:00.980Z", "content": "

bro people from hugginface , mistral ai , what do you expect ??

", "rawMarkdown": "bro people from hugginface , mistral ai , what do you expect ??\n", "votes": 5, "replies": [ { "id": 2916545, "postDate": "2024-07-11T04:54:15.713Z", "content": "

Wait i didn't know that…

", "rawMarkdown": "Wait i didn't know that...", "votes": 1 } ] } ] }, { "id": 3426769, "postDate": "2026-03-23T07:36:49.283Z", "content": "

i understood

", "rawMarkdown": "i understood\n" }, { "id": 3197258, "postDate": "2025-05-08T01:18:52.557Z", "content": "

Where there is a will, there is a way.

", "rawMarkdown": "Where there is a will, there is a way." }, { "id": 3088269, "postDate": "2025-01-04T13:41:08.030Z", "content": "

why you chose fine-tune DeepSeekMath-Base 7B there is any specific reason, because you have 2 or more option rho,  qwen ect? 

", "rawMarkdown": "why you chose fine-tune DeepSeekMath-Base 7B there is any specific reason, because you have 2 or more option rho,  qwen ect?  " }, { "id": 3037274, "postDate": "2024-11-05T14:53:07.980Z", "content": "

I loved this notebook. It is very informative especially for the Progress Prize 2. Thank you so much project Numina

", "rawMarkdown": "I loved this notebook. It is very informative especially for the Progress Prize 2. Thank you so much project Numina" }, { "id": 2932742, "postDate": "2024-07-23T07:55:03.533Z", "content": "

Congrats on the 1st place. Insightful read, thanks for sharing!

", "rawMarkdown": "Congrats on the 1st place. Insightful read, thanks for sharing! " }, { "id": 2926482, "postDate": "2024-07-17T18:36:52.957Z", "content": "

Congrats! Super impressive :)

", "rawMarkdown": "Congrats! Super impressive :)" }, { "id": 2924765, "postDate": "2024-07-16T17:53:05.097Z", "content": "

Impressive work

", "rawMarkdown": "Impressive work" }, { "id": 2924393, "postDate": "2024-07-16T13:43:46.570Z", "content": "

Thanks for the solution description, could you elaborate more on why you were not confident on LoRA?

\n
\n

In both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation.

\n
", "rawMarkdown": "Thanks for the solution description, could you elaborate more on why you were not confident on LoRA? \n\n> In both stages, we performed “full fine-tuning” where all weights of the model are updated during backpropagation. In other words, we did not use parameter efficient techniques like LoRA or DoRA because we were not confident they could match the performance of full fine-tuning without significant experimentation.", "replies": [ { "id": 2925021, "postDate": "2024-07-16T21:22:07.673Z", "content": "

It's mostly informed by our experience training chat models like Zephyr where we saw LoRA can match the performance of full training, but at the expense of adding new hyperparameters to the mix (rank & alpha) which require more time to get right. We think DoRA (or QDoRA) should work better, but we have not tinkered with it yet and were already tight for time. This blog post from answer.ai has some nice details on what it takes to make QDoRA work: http://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html

", "rawMarkdown": "It's mostly informed by our experience training chat models like Zephyr where we saw LoRA _can_ match the performance of full training, but at the expense of adding new hyperparameters to the mix (rank & alpha) which require more time to get right. We think DoRA (or QDoRA) should work better, but we have not tinkered with it yet and were already tight for time. This blog post from answer.ai has some nice details on what it takes to make QDoRA work: http://www.answer.ai/posts/2024-04-26-fsdp-qdora-llama3.html", "votes": 7, "replies": [ { "id": 2925402, "postDate": "2024-07-17T06:51:28.130Z", "content": "

Thanks for the reply and the link, it's a very nice read.

", "rawMarkdown": "Thanks for the reply and the link, it's a very nice read. " } ] } ] }, { "id": 2921414, "postDate": "2024-07-14T11:18:59.690Z", "content": "

Congratulations and thaks for sharing impressive solution. It was very interesting.

\n

I think code execution feedback will be the key trend for better quality synthetic data generation, especially effective for math and coding domains.

", "rawMarkdown": "Congratulations and thaks for sharing impressive solution. It was very interesting.\n\nI think code execution feedback will be the key trend for better quality synthetic data generation, especially effective for math and coding domains." }, { "id": 2918149, "postDate": "2024-07-12T03:54:46.117Z", "content": "

Congratulations on winning this competition. Thanks for sharing the details of your solution.

", "rawMarkdown": "Congratulations on winning this competition. Thanks for sharing the details of your solution. " }, { "id": 3078837, "postDate": "2024-12-22T22:51:51.157Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2915681, "postDate": "2024-07-10T15:53:12.393Z", "rawMarkdown": "", "votes": 4, "isDeleted": true } ], "complete": true, "source": "live", "fetched_at": "2026-10-20T06:55:00Z", "competition_id": 73231, "competition_slug": "ai-mathematical-olympiad-prize", "team_id": 11912468, "solution_url": "/competitions/ai-mathematical-olympiad-prize/writeups/numina-numina-1st-place-solution" }