# 627th Solution for CommonLit ESS competition- What doesn't kill you makes you stronger

This was the first NLP task & first Kaggle competition that I worked really hard on.
Even though my final result is not satisfying, I have learned a lot through the journey.
For those who didn't join the competition.
You can check the competition overview
[here](https://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/overview)
.
You can check the competition dataset
[here](https://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/data)
.
Overview of Approach
Spell Checking Tools
In the beginning, I started off by modifying the notebook created by other competitors (
debertav3base
).
Then, I realized that I wouldn’t be developing myself in this field just by copying others’ work. (I had to
copy & edit
later on anyway, trying to achieve high score)
So, I started off by developing my own preprocessing method, focusing on
[misspelled words](https://www.kaggle.com/code/jasonheesanglee/eda-spellcheck-tool-comparison)
.
Here, I found that SymSpell worked better on the dataset we have than any other libraries.
Later (now), I understood that this wasn’t the key to this competition.
MobileBERT
After taking a closer look at the Competition Overview, I decided to try out
MobileBERT
with the concept of helping teachers located in areas where there is less or no internet connection.
My journey with
MobileBERT
is explained in this
[notebook](https://www.kaggle.com/code/jasonheesanglee/en-mobilebert-implementation)
.
TL;DR
I first tried to apply
MobileBERT
on the code that I was working on with
debertav3base
.
However, it didn’t work.
After reading the
[research paper](assets/ec52fa5b8b9a2d53916a60e1.pdf)
, I understood that the teacher model was needed to educate MobileBERT on the task.
However, in terms of the competition, it didn’t give a good result.
All models
So I had to move back to
debertav3base
+
LGBM
.
I finally got an OK score, but I couldn’t stop trying other methods. (Check this
[notebook](https://www.kaggle.com/code/jasonheesanglee/lgbm-autocorrect-allmodels)
for my record)
So, I have tried all different models in this
[Dataset](https://www.kaggle.com/datasets/kozodoi/transformers)
.
Most models worked fine on the logic I had but didn’t give a good score as expected. (There should be reason behind why other people don’t go with haha).
The only thing I regret is that I couldn’t try with
T5
, which didn’t work on my logic (Of course it wouldn’t work.)
Modifying Preprocessing
In the last few days, I tried to focus on developing the preprocessing part & the
LGBM
part of the code.
For the preprocessing part, I have
[separated this part](https://www.kaggle.com/code/jasonheesanglee/datapreprocess)
from the submission notebook to save some time with submission and added SymSpell as a spell-checking tool.
And Optuna for finding the best parameter for LGBM.
It worked fine on the Public Leaderboard (0.44) but unfortunately didn’t work on the Private Leaderboard.
At the end of the day, I am still alive, learned a lot through this competition.
On top of that, I am happy that I have more opportunities to challenge myself!