# 79th place Solution

Thank you for hosting this competition.
I'm so excited to have finished my first Kaggle competition with a silver medal!
Thanks to shake up we made our own medal.
The main techiques are not different as other well organized public notebooks.
We took our baseline from the public notebook made by
[@tsunotsuno](https://www.kaggle.com/tsunotsuno)
.
Special thanks to the best contributor of this competition.
We made a little changes though, which are addressed below briefly.
What Worked
deberta-v3-large + LGBM
re-init last 2 layers
replace '\n\n', '\r\n' with [PARAGRAPH]
Use Optuna
nelder-mead for finding Ensemble weights
Get target-wised Model
Post-processing(low impact, private score : + 0.0001)
We took a look at the preprocessed data and found some patterns from it.
So we arranged the content and wording score like this based on what we analyzed
test.loc[(test[]>)&(test[]>-), ] = -
test.loc[(test[]>)&(test[]<=)&(test[]>), ] = 
test.loc[(test[]>)&(test[]>-), ] = -
test.loc[(test[]>)&(test[]<=)&(test[]>-), ] = -
test.loc[(test[]>)&(test[]<=)&(test[]>), ] =
What Didn't Work
freezing n layers
MeanPooling, AttentionPooling
change batch size
Models other than deberta-v3-large (deberta-v2-xlarge, albert-large-v2, muppet-roberta-large, bigbird-roberta-large, roberta-large)
Thanks again to the competition hosts and everyone who participated!! Good luck for your next competitions!
Team member
[@kimseunghee](https://www.kaggle.com/kimseunghee)
[@danielchae](https://www.kaggle.com/danielchae)