# 142nd place solution

First of all, congratulations to all the winners. I am gonna describe all the things that we tried and what worked for us and what didn't.
What didn't work in our case?
1.  We experimented with a lot of different models. Starting from ResNet-50 to get a baseline, we tried all variants of ResNets including ResNeXt but none of them gave us a good score on the leaderboard.
2. We approached the problem from both ends. I tried solving it as a classification problem while
[Konrad](https://www.kaggle.com/konradb)
and
[Abhishek](https://www.kaggle.com/abhishek)
approached it as a regression problem. After a while, I realized that treating it as a classification problem isn't providing a boost to the score.
3. Ben's preprocessing: In fact, all models where we used this resulted in a very low score.
4. Label smoothing didn't work very well for classification models
What worked for us?
1. For preprocessing, we found out that cropping out the black area in the original images is good enough.
2. Heavy augmentation: Random horizontal and vertical flips weren't enough.
3.
EfficientNets
, especially
B4
and
B5
, proved more effective than anything else. We ended up using
B4
only
4. We used both
old competition
data as well as
new competition
data but not blindly. Both the datasets have huge
class imbalance
with classes
0
and
2
as majority classes. We tried to balance the classes as much as possible.
One crazy idea
For the last attempt, we tried a crazy idea regarding
sampling
of the classes from the old data. We were positive that it would work but we weren't very sure at any point in time.  We randomly sampled the old dataset in a fashion such that the class distribution for old dataset looked like this:
3    2087
4    1914
2    1000
1    1000
0    1000
We didn't want to throw away any data point in the new dataset. So, we
undersampled
the classes that were in
majority
in the new dataset. Doing this would induce a bias in the model during
pretraining
phase towards the classes
3
and
4
. Once we switch to
fine-tuning
the model on the new dataset, the distribution is reversed now and the model should be able to correct the bias. With this distribution and heavy augmentation, our best model,
EfficientNet-B4
, with 5-folds TTA scored
0.920
on the final leaderboard.
What we didn't try?
1. We didn't try pseudo labelling. 
2. We didn't ensemble our models, especially all variants of
EfficientNets
.
We didn't try these things because during the last month because we all three were having too much work on our end. I was diagnosed with
dengue
and couldn't contribute fully in between. Nevertheless, we are happy that with a
single model
we made a jump of
655
positions on the leaderboard and didn't end badly. Thanks to my amazing teammates @konradb and @abhishek