# [9th place] Short summary

Thanks to the hosts for this interesting competition and
[@christofhenkel](https://www.kaggle.com/christofhenkel)
for the fun team!
We are happy to make the jump on private LB which is as often is the case based on a strong blend that is not overfitting the public LB. That said, our best sub is both our best pub LB and private LB and we made the best selection.
Image norm
We observed slight differences in image channel distributions between train and test and doing local image normalization brought CV and LB closer together for us, which is why we sticked to it throughout our final models. That means we standardize each channel locally for each image.
Augmentations
Most models only use standard flips and transposes. Some models also add cutout and some add tiny random noise. For TTA we do either TTA4 or TTA8.
Models
We only use EfficientNets. We started fitting models only on fold 0, but then switched to fitting models "blindly" on full data as CV convergence always was stable for us. If we were unsure, we re-fitted on a fold on a new model and checked if the models overfit or not, and then trusted the full fits. This also meant that we had to blend a bit blindly though.
We both fitted vanilla EfficientNets, but also got huge boosts on CV when changing the stride in the first layer to (1,1), as also other contestants did. This means that the models are fit for longer on the full resolution, and this is specifically helpful as a lot of information about the manipulation lies between neighboring pixels. We observed this facet also when randomly shuffling neighboring pixels in TTA and as soon as you shuffle further than one pixel away, the score deteriorates heavily.
Our final sub consists of the following models:
Vanilla EfficientNet B6 (one fold)
Vanilla EfficientNet B8 (full data)
Stride 1 EfficientNet B1 (full data)
Stride 1 EfficientNet B2 (full data)
Stride 1 EfficientNet B3 (full data)
Unfortunately, we dont have the full logs plotted as some models finished on the last day, but this is how the training scores across models roughly look like (2x refers to stride 1 models):
![](https://i.imgur.com/ooXn6U9.png)
You can see that B3 Stride 1 model was the best, also on a separate fit on CV. However, vanilla B6 model was the best on public LB, so things are a bit misleading there. On private LB actually B3 Stride 1 alone is our best one (925), followed by B8 (924) and so on. So actually matching the ranking on the plot above.
Fitting
All models are fitted over 40 epochs with cosine decay, AdamW, mixed precision, and varying batch sizes. We did not see changes in BS and LR to have much impact, which is why we basically left it untouched across models as we also did not have the time to tune it further.
Blending
Our final blend is composed of a 50-50 blend between (1) a blend between vanilla B6 and B8 (LB Score 936) and (2) a blend between epochs 30,35,39 of the B1,B2,B3 Stride 1 models (LB Score 935). The blend scored 937 on public LB and 928 on private LB.