# 3rd place solution

Hi kaggler!
I only used 5 hours in this competition.
After spending those 5 hours, I realized that I couldn't receive prize money due to my nationality, so I gave up lol
1. What I tried
[reranker](https://www.kaggle.com/code/aleaiest/bge-reranker-v2-m3)
: The performance was quite poor. Even with fine-tuning, I don't think it would have performed well (BAAI/bge-reranker-v2-m3: 0.1954)
[llm logit processor](https://www.kaggle.com/code/shionao7/kac-qwen2-5-32b-only-use-test-data-please-upvote)
: I was inspired by this notebook. I tried both Qwen 3 unreasoning and Qwen 2.5, but since Qwen 2.5 seemed to have better baseline performance, I proceeded with fine-tuning it using axolotl.
Here's an example prompt:
def create_prompt(text, answer):
    user_prompt = f

    messages = [
        {: , : },
        {: , : user_prompt},
        {: , : str(answer)}
I fine-tuned with a learning rate of 1e-3 using the above prompt through axolotl, and then performed
[GPTQ quantization](https://github.com/ModelCloud/GPTQModel)
.
For calibration data, I selected 1,280 samples from the training data.
As those who have seen the data would know, most token lengths are less than 300!
[This is my train & quantization code](https://www.kaggle.com/code/aleaiest/3rd-qwen2-5-32b-train-quantization-code)
[This is my final kaggle inference notebook](https://www.kaggle.com/code/aleaiest/kac-qwen3-32b-finetuning)
2. What I was planning to try next
Extract only the answer through reasoning using the Qwen 3 model without fine-tuning
3. ETC
Thanks for reading!
This is my first solution on kaggle, so if you have any additional questions, please leave a comment.
Collaboration are also welcome.