# 4th place solution (2nd after LB cleaning)

Congrats everyone with excellent results and ending of this exciting competition!
Data
We used full-size images from 2015 train dataset to pretrain our models. Models trained on old data only gave us ~0.73-0.75 on public LB.
Current competition train.
Old competition test.
Models
Overall we tried different architectures including se-resnext50, se-resnext101, densenet, but they were slow to train and performed poorly. The only NN architecture that performed well was efficient-net, so our final solution contains multiple b3, b4, and b5 networks.
B3 image size: 300
B4 image size: 460
B5 image size: 456
Preprocessing
I cropped black background on images and then resized to the desired image size before training my models.
[@aispiriants](/aispiriants)
didn't perform any preprocessing except resizing images. I've also tried Ben's color preprocessing, but it didn't boost score, so I stop trying in the middle of the competition.
Augmentations
We used a lot of augmentations, at least more than I ever used before :) 
All from the wonderful albumentations library: Blur, Flip, RandomBrightnessContrast, ShiftScaleRotate, ElasticTransform, Transpose, GridDistortion, HueSaturationValue, CLAHE, CoarseDropout.
Training
Overall training/submit flow is pretty simple:
Pretrain on old data (~80 epochs on b3, ~15 epochs on b5)
Train on current competition data (~50 epochs on b3, ~15 epochs on b5)
Pick top-performing epochs from the training process and blend them by a simple mean. We also used flips as TTA.
Secret ingredient :)
We pseudo-labeled current test data and then fine-tune networks again using train+pseudolabeled test. It gave us a huge boost and a lot of motivation, so we repeated a pseudo-labeling round once more that boosted a little bit more. We also pseudo-labeled previous competition test data to increase our training dataset size and fine-tuned some models on it too.