{
  "id": 578580,
  "title": "3rd place solution",
  "content": "<h1>Hi kaggler!</h1>\n<p>I only used 5 hours in this competition.<br>\nAfter spending those 5 hours, I realized that I couldn't receive prize money due to my nationality, so I gave up lol</p>\n<h1>1. What I tried</h1>\n<p><a href=\"https://www.kaggle.com/code/aleaiest/bge-reranker-v2-m3\" target=\"_blank\">reranker</a>: The performance was quite poor. Even with fine-tuning, I don't think it would have performed well (BAAI/bge-reranker-v2-m3: 0.1954)<br>\n<a href=\"https://www.kaggle.com/code/shionao7/kac-qwen2-5-32b-only-use-test-data-please-upvote\" target=\"_blank\">llm logit processor</a>: I was inspired by this notebook. I tried both Qwen 3 unreasoning and Qwen 2.5, but since Qwen 2.5 seemed to have better baseline performance, I proceeded with fine-tuning it using axolotl.<br>\nHere's an example prompt:</p>\n<pre><code>def create_prompt(text, answer):\n    user_prompt = f\n\n    messages = [\n        {: , : },\n        {: , : user_prompt},\n        {: , : str(answer)}\n</code></pre>\n<p>I fine-tuned with a learning rate of 1e-3 using the above prompt through axolotl, and then performed <a href=\"https://github.com/ModelCloud/GPTQModel\" target=\"_blank\">GPTQ quantization</a>.<br>\nFor calibration data, I selected 1,280 samples from the training data.<br>\nAs those who have seen the data would know, most token lengths are less than 300!<br>\n<a href=\"https://www.kaggle.com/code/aleaiest/3rd-qwen2-5-32b-train-quantization-code\" target=\"_blank\">This is my train &amp; quantization code</a><br>\n<a href=\"https://www.kaggle.com/code/aleaiest/kac-qwen3-32b-finetuning\" target=\"_blank\">This is my final kaggle inference notebook</a></p>\n<h1>2. What I was planning to try next</h1>\n<p>Extract only the answer through reasoning using the Qwen 3 model without fine-tuning</p>\n<h1>3. ETC</h1>\n<p>Thanks for reading!<br>\nThis is my first solution on kaggle, so if you have any additional questions, please leave a comment.<br>\nCollaboration are also welcome. </p>",
  "comments": [
    {
      "id": 3201753,
      "author_name": "Denver Magtibay",
      "author_url": "",
      "post_date": "2025-05-14T11:02:49.440000",
      "content": "<p>Congrats sir! Great technique!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3202159,
          "author_name": "Taewoong Kim",
          "author_url": "",
          "post_date": "2025-05-15T00:12:39.867000",
          "content": "<p>You're welcome, thx :)</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3200166,
      "author_name": "Taewoong Kim",
      "author_url": "",
      "post_date": "2025-05-12T07:11:31.570000",
      "content": "<p>I just found out they're giving me a prize too!<br>\nThank you so much for organizing such a great contest!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3200177,
          "author_name": "Mohamed Eltayeb",
          "author_url": "",
          "post_date": "2025-05-12T07:40:05.913000",
          "content": "<p>Well, getting into 3rd place using only 2 subs really deserves a prize :)<br>\nWell done!</p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "comment_count": 4,
  "messages": [
    {
      "id": 3200042,
      "postDate": "2025-05-11T23:42:07.610Z",
      "content": "<h1>Hi kaggler!</h1>\n<p>I only used 5 hours in this competition.<br>\nAfter spending those 5 hours, I realized that I couldn't receive prize money due to my nationality, so I gave up lol</p>\n<h1>1. What I tried</h1>\n<p><a href=\"https://www.kaggle.com/code/aleaiest/bge-reranker-v2-m3\" target=\"_blank\">reranker</a>: The performance was quite poor. Even with fine-tuning, I don't think it would have performed well (BAAI/bge-reranker-v2-m3: 0.1954)<br>\n<a href=\"https://www.kaggle.com/code/shionao7/kac-qwen2-5-32b-only-use-test-data-please-upvote\" target=\"_blank\">llm logit processor</a>: I was inspired by this notebook. I tried both Qwen 3 unreasoning and Qwen 2.5, but since Qwen 2.5 seemed to have better baseline performance, I proceeded with fine-tuning it using axolotl.<br>\nHere's an example prompt:</p>\n<pre><code>def create_prompt(text, answer):\n    user_prompt = f\n\n    messages = [\n        {: , : },\n        {: , : user_prompt},\n        {: , : str(answer)}\n</code></pre>\n<p>I fine-tuned with a learning rate of 1e-3 using the above prompt through axolotl, and then performed <a href=\"https://github.com/ModelCloud/GPTQModel\" target=\"_blank\">GPTQ quantization</a>.<br>\nFor calibration data, I selected 1,280 samples from the training data.<br>\nAs those who have seen the data would know, most token lengths are less than 300!<br>\n<a href=\"https://www.kaggle.com/code/aleaiest/3rd-qwen2-5-32b-train-quantization-code\" target=\"_blank\">This is my train &amp; quantization code</a><br>\n<a href=\"https://www.kaggle.com/code/aleaiest/kac-qwen3-32b-finetuning\" target=\"_blank\">This is my final kaggle inference notebook</a></p>\n<h1>2. What I was planning to try next</h1>\n<p>Extract only the answer through reasoning using the Qwen 3 model without fine-tuning</p>\n<h1>3. ETC</h1>\n<p>Thanks for reading!<br>\nThis is my first solution on kaggle, so if you have any additional questions, please leave a comment.<br>\nCollaboration are also welcome. </p>",
      "rawMarkdown": "# Hi kaggler!\nI only used 5 hours in this competition.\nAfter spending those 5 hours, I realized that I couldn't receive prize money due to my nationality, so I gave up lol\n\n# 1. What I tried\n[reranker](https://www.kaggle.com/code/aleaiest/bge-reranker-v2-m3): The performance was quite poor. Even with fine-tuning, I don't think it would have performed well (BAAI/bge-reranker-v2-m3: 0.1954)\n[llm logit processor](https://www.kaggle.com/code/shionao7/kac-qwen2-5-32b-only-use-test-data-please-upvote): I was inspired by this notebook. I tried both Qwen 3 unreasoning and Qwen 2.5, but since Qwen 2.5 seemed to have better baseline performance, I proceeded with fine-tuning it using axolotl.\nHere's an example prompt:\n```\ndef create_prompt(text, answer):\n    user_prompt = f\"\"\"You are an expert math problem classifier.\n\n    Task: Classify the following math problem into one of the categories below.\n    Instruction: Respond with the correct number (0-7) only. Do NOT include any explanation.\n\n    Math Problem:\n    {text}\n\n    Categories:\n    0. Algebra  \n    1. Geometry and Trigonometry  \n    2. Calculus and Analysis  \n    3. Probability and Statistics  \n    4. Number Theory  \n    5. Combinatorics and Discrete Math  \n    6. Linear Algebra  \n    7. Abstract Algebra and Topology\n\n    Answer (number only): \"\"\"\n    \n    messages = [\n        {\"role\": \"system\", \"content\": \"You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\"},\n        {\"role\": \"user\", \"content\": user_prompt},\n        {\"role\": \"assistant\", \"content\": str(answer)}\n```\n\nI fine-tuned with a learning rate of 1e-3 using the above prompt through axolotl, and then performed [GPTQ quantization](https://github.com/ModelCloud/GPTQModel).\nFor calibration data, I selected 1,280 samples from the training data.\nAs those who have seen the data would know, most token lengths are less than 300!\n[This is my train & quantization code](https://www.kaggle.com/code/aleaiest/3rd-qwen2-5-32b-train-quantization-code)\n[This is my final kaggle inference notebook](https://www.kaggle.com/code/aleaiest/kac-qwen3-32b-finetuning)\n\n# 2. What I was planning to try next\nExtract only the answer through reasoning using the Qwen 3 model without fine-tuning\n\n# 3. ETC\nThanks for reading!\nThis is my first solution on kaggle, so if you have any additional questions, please leave a comment.\nCollaboration are also welcome. ",
      "votes": 11
    },
    {
      "id": 3201753,
      "postDate": "2025-05-14T11:02:49.440Z",
      "content": "<p>Congrats sir! Great technique!</p>",
      "rawMarkdown": "Congrats sir! Great technique!",
      "votes": 1,
      "replies": [
        {
          "id": 3202159,
          "postDate": "2025-05-15T00:12:39.867Z",
          "content": "<p>You're welcome, thx :)</p>",
          "rawMarkdown": "You're welcome, thx :)"
        }
      ]
    },
    {
      "id": 3200166,
      "postDate": "2025-05-12T07:11:31.570Z",
      "content": "<p>I just found out they're giving me a prize too!<br>\nThank you so much for organizing such a great contest!</p>",
      "rawMarkdown": "I just found out they're giving me a prize too!\nThank you so much for organizing such a great contest!",
      "votes": 1,
      "replies": [
        {
          "id": 3200177,
          "postDate": "2025-05-12T07:40:05.913Z",
          "content": "<p>Well, getting into 3rd place using only 2 subs really deserves a prize :)<br>\nWell done!</p>",
          "rawMarkdown": "Well, getting into 3rd place using only 2 subs really deserves a prize :)\nWell done!",
          "votes": 2
        }
      ]
    }
  ],
  "complete": true,
  "source": "live",
  "fetched_at": "2026-10-19T22:52:56Z",
  "competition_id": 97669,
  "competition_slug": "classification-of-math-problems-by-kasut-academy",
  "team_id": 13788862,
  "solution_url": "/competitions/classification-of-math-problems-by-kasut-academy/writeups/3rd-place-solution"
}