# 167th Place Solution [Deberta Large with LoRA]

Submission Overview
Models
CV
Public
Private
Deberta base + lgb
508
447
485
Deberta large LoRA + lgb
501
441
495
Result blend + lgb
492
435
482
Things which didn't work
Fine-tuning like GPT models for regression
SFT Llama for direct target generation
Additional data from ChatGPT, SFT Llama2
Example code how to fine-tuning with LoRA
[fine-tuning-deberta-large-with-lora-adapter](https://www.kaggle.com/code/alexcei/fine-tuning-deberta-large-with-lora-adapter)