You are prohibited to use those super large LLMs (> 4B) to perform on this Kaggle contest. You are allowed to use 1B-3B scale models freely with different finetuning strategies.