# Private LB 105th / Public LB 970th solution and first medal 🥳

First of all, congratulations to all the competitors. Huge thanks to Kaggle, Makerere University AI Lab for organising this competition.
I'd also like to thank my amazing teammate,
[@kjkr73](https://www.kaggle.com/kjkr73)
. He did most of the heavy lifting. He'll be publishing a kernel soon.
⚠  We actually jumped to 105. :)
Data
We only used the competition data.
Validation Strategy
Stratified 5-fold.
My CV score and LB score were correlated from the very start so I didn't worry about it too much.
Trusted our CV but didn't disregard Public LB entirely.
Image Preprocessing And Augmentations
We trained models on 224*224, 256*256, 384*384 and 512*512 image size. Larger image size always had better scores.
We tried multiple combinations of augmentations. Following combinations performed the best:
Cutout, HorizontalFlip, VerticalFlip, ShiftScaleRotate, HueSaturationValue, RandomBrightnessContrast.
Cutout, CoarseDropout,  HorizontalFlip, VerticalFlip.
Neural Network Architectures
We tried ResNet34, ResNext50_32x4d, EfficientNet B1 NS, EfficientNet B3 NS, EfficientNet B4 NS, ViT-Base patch 16, and DeiT-Base patch 16 224.
Miscellaneous
Loss(es):
Bi-tempered loss with label smoothing
Taylor Crossentropy with label smoothing
Optimizer:
Adam
LR Scheduler(s):
OneCycleLR
CosineAnnealingWarmRestarts
Hardware/ platform:
Kaggle kernels
Final Submissions
Selected Submission 1 (Private 0.8999, Public 0.9009) is a blend of:
EfficientNet B1 NS on 512*512 image size [1xTTA].
ResNext50_32x4d on 512*512 image size [1xTTA].
ResNext50_32x4d on 512*512 image size [No TTA].
EfficientNet B3 NS on 512*512 image size [No TTA].
Selected Submission 2 (Private 0.8981, Public 0.9002) is a blend of:
EfficientNet B1 NS on 512*512 image size [1xTTA].
ResNext50_32x4d on 512*512 image size [No TTA].
EfficientNet B3 NS on 512*512 image size [No TTA].
Some observations/ tips for competing in a CV comp with Kaggle Kernels only:
Data preprocessing != data augmentation. Resize images to the intended size before training, it will save you some GPU quota and speed up your experiments. Credits:
[@cdeotte](https://www.kaggle.com/cdeotte)
; read more
[here](https://www.kaggle.com/c/siim-isic-melanoma-classification/discussion/161446)
.
Create a proof of concept kernel which is essentially a smaller version of the training kernel for fast experimentation or to check the feasibility of an idea. This kernel should run quickly and return an OOF CV score. You can make this kernel fast with a smaller number of CV folds/ smaller number of epochs. If you see improvement in this CV score, copy the config and run it on your training kernel.
Write modular, maintainable, and readable code.
Credits
pandas
[https://github.com/pandas-dev/pandas](https://github.com/pandas-dev/pandas)
torch
[https://github.com/pytorch/pytorch](https://github.com/pytorch/pytorch)
timm
[https://github.com/rwightman/pytorch-image-models](https://github.com/rwightman/pytorch-image-models)
.
albumentations
[https://github.com/albumentations-team/albumentations](https://github.com/albumentations-team/albumentations)
lycon
[https://github.com/ethereon/lycon](https://github.com/ethereon/lycon)
PS: This is our (my and
[@kjkr73](https://www.kaggle.com/kjkr73)
's) first medal 🥳
Edit: Updated LB scores and rank.