# 5th place solution

Congrats to everyone,  Here is my solution, not complicated.
models
efficientnet-b0
efficientnet-b1
efficientnet-b2
efficientnet-b3
I just combined the classification and regression task together, multi loss maybe not easy to ovetfitting. But the test phrase just with the prediction of regression, code like these
![](assets/7d671d619d385a29c415b478.jpg)
Data preprocessing and Augmentations
No more special preprocessing method, I just cropped black background on images.
image_size: 384x384
Just Fliplr, Flipud, Randomrotate(0, 360), zoom(1.0, 1.35) and self designed perspective transformation method.
Training strategy
5 fold CV
according to the better public LB, I did not pretrained on 2015-data and then finetune on 2019-data, I just combined 2015-data and 2019-data together and then train the model, valid on 2015 priavte test data and 2019 valid data.
lr_schedule: adam optimizer, lrs=[5e-4, 1e-4, 1e-5, 1e-6], and the position of lr decays is [10, 16, 22], train 25 epochs.
2*TTA
threshold: [0.5, 1.5, 2.5, 3.5]
Pseudo labels
I think the pseudo labels is the most import key to win in this competition. What I thinking about is, because we don't know the distribution of the 2019 private test set, it might be similar with 2015data, or 2019 training data or 2019 public data. So I can combine three different distribution data together to train the model,  and the model can recognition three different distribution data.
I don't use all the 2019 public data as the pseudo labels samples, I chose the higer confidence samples, the codes like these
![](assets/c713f67fe0e37198c5562a0e.jpg)
pseudo labels helpd me improve the local mse loss and qwk.
Conclusion
the trainning data is strange and most kagglers met some strange problems in this competition, like the LB is unstable and can not find the relation between the local cv and public LB, and...
But the final results show that we can still trust our local CV.