This was the first NLP task & first Kaggle competition that I worked really hard on.
\nEven though my final result is not satisfying, I have learned a lot through the journey.
\n
You can check the competition overview here.
\nYou can check the competition dataset here.
In the beginning, I started off by modifying the notebook created by other competitors (debertav3base).
\nThen, I realized that I wouldn’t be developing myself in this field just by copying others’ work. (I had to copy & edit later on anyway, trying to achieve high score)
\nSo, I started off by developing my own preprocessing method, focusing on misspelled words.
\nHere, I found that SymSpell worked better on the dataset we have than any other libraries.
\nLater (now), I understood that this wasn’t the key to this competition.
\n
After taking a closer look at the Competition Overview, I decided to try out MobileBERT with the concept of helping teachers located in areas where there is less or no internet connection.
\nMy journey with MobileBERT is explained in this notebook.
\nTL;DR
\nI first tried to apply MobileBERT on the code that I was working on with debertav3base.
\nHowever, it didn’t work.
\nAfter reading the research paper, I understood that the teacher model was needed to educate MobileBERT on the task.
\nHowever, in terms of the competition, it didn’t give a good result.
\n
So I had to move back to debertav3base + LGBM.
\nI finally got an OK score, but I couldn’t stop trying other methods. (Check this notebook for my record)
\nSo, I have tried all different models in this Dataset.
\nMost models worked fine on the logic I had but didn’t give a good score as expected. (There should be reason behind why other people don’t go with haha).
\nThe only thing I regret is that I couldn’t try with T5, which didn’t work on my logic (Of course it wouldn’t work.)
\n
In the last few days, I tried to focus on developing the preprocessing part & the LGBM part of the code.
\nFor the preprocessing part, I have separated this part from the submission notebook to save some time with submission and added SymSpell as a spell-checking tool.
\nAnd Optuna for finding the best parameter for LGBM.
\nIt worked fine on the Public Leaderboard (0.44) but unfortunately didn’t work on the Private Leaderboard.
\n
At the end of the day, I am still alive, learned a lot through this competition.
\nOn top of that, I am happy that I have more opportunities to challenge myself!
Same, glad that you share your passion and work with us.
", "votes": 2, "replies": [] }, { "id": 2478431, "author_name": "Shreyansh Murathia", "author_url": "", "post_date": "2023-10-12T02:32:57.080000", "content": "Great job! I wish you a successful future…
", "votes": 2, "replies": [ { "id": 2478432, "author_name": "Jason Heesang Lee", "author_url": "", "post_date": "2023-10-12T02:34:43.880000", "content": "Thank you @shreyanshmurathia for the wish!!
", "votes": 0, "replies": [] } ] } ], "comment_count": 3, "messages": [ { "id": 2478426, "postDate": "2023-10-12T02:21:57.760Z", "content": "This was the first NLP task & first Kaggle competition that I worked really hard on.
\nEven though my final result is not satisfying, I have learned a lot through the journey.
\n
You can check the competition overview here.
\nYou can check the competition dataset here.
In the beginning, I started off by modifying the notebook created by other competitors (debertav3base).
\nThen, I realized that I wouldn’t be developing myself in this field just by copying others’ work. (I had to copy & edit later on anyway, trying to achieve high score)
\nSo, I started off by developing my own preprocessing method, focusing on misspelled words.
\nHere, I found that SymSpell worked better on the dataset we have than any other libraries.
\nLater (now), I understood that this wasn’t the key to this competition.
\n
After taking a closer look at the Competition Overview, I decided to try out MobileBERT with the concept of helping teachers located in areas where there is less or no internet connection.
\nMy journey with MobileBERT is explained in this notebook.
\nTL;DR
\nI first tried to apply MobileBERT on the code that I was working on with debertav3base.
\nHowever, it didn’t work.
\nAfter reading the research paper, I understood that the teacher model was needed to educate MobileBERT on the task.
\nHowever, in terms of the competition, it didn’t give a good result.
\n
So I had to move back to debertav3base + LGBM.
\nI finally got an OK score, but I couldn’t stop trying other methods. (Check this notebook for my record)
\nSo, I have tried all different models in this Dataset.
\nMost models worked fine on the logic I had but didn’t give a good score as expected. (There should be reason behind why other people don’t go with haha).
\nThe only thing I regret is that I couldn’t try with T5, which didn’t work on my logic (Of course it wouldn’t work.)
\n
In the last few days, I tried to focus on developing the preprocessing part & the LGBM part of the code.
\nFor the preprocessing part, I have separated this part from the submission notebook to save some time with submission and added SymSpell as a spell-checking tool.
\nAnd Optuna for finding the best parameter for LGBM.
\nIt worked fine on the Public Leaderboard (0.44) but unfortunately didn’t work on the Private Leaderboard.
\n
At the end of the day, I am still alive, learned a lot through this competition.
\nOn top of that, I am happy that I have more opportunities to challenge myself!
Same, glad that you share your passion and work with us.
", "rawMarkdown": "Same, glad that you share your passion and work with us.", "votes": 2 }, { "id": 2478431, "postDate": "2023-10-12T02:32:57.080Z", "content": "Great job! I wish you a successful future…
", "rawMarkdown": "Great job! I wish you a successful future...", "votes": 2, "replies": [ { "id": 2478432, "postDate": "2023-10-12T02:34:43.880Z", "content": "Thank you @shreyanshmurathia for the wish!!
", "rawMarkdown": "Thank you @shreyanshmurathia for the wish!!" } ] } ], "complete": true, "source": "live", "fetched_at": "2026-10-19T23:00:42Z", "competition_id": 53482, "competition_slug": "commonlit-evaluate-student-summaries", "team_id": 10720678, "solution_url": "/competitions/commonlit-evaluate-student-summaries/writeups/jason-heesang-lee-627th-solution-for-commonlit-ess" }