# 78th place solution (silver medal)

Congrats to the top solutions and a great competition.
One learns new thing in every competition, a perfect way to follow the journey of AI world and also to contribute to important problems, thanks Kaggle, competition host and fellow kagglers.
The solution was a global ensemble of 29 different models including 4 local ensembles(2 own, 2 public), trained in different ways.
All trained with dim 512 but the rest I tried using different techniques and parameters.
Own local trainings and ensembles
1 - FastAIv2 - 6 models
Common for all training
Used 1-fold of each model from random 10-fold training.
Ranger or AdamW optimizer
LabelSmoothingCrossEntropy loss
1CycleScheduler
Started all training with 3 epochs with only training last layers and then trained all layers 20 epochs.
In between the trainings I used my own custom LR finder, which I posted in another competition.
“LR Finder alternative use.”
[https://www.kaggle.com/c/jane-street-market-prediction/discussion/218711](https://www.kaggle.com/c/jane-street-market-prediction/discussion/218711)
Models
2 tf_efficientnet_b5_ns – mixup training
2 seresnext50_32x4d – cutmix training
1 xception – cutmix training – here I also used upsample with more upsample to class 4 which I visually noticed had more false classifications.
1 densenet121 – cutmix training – here I also used upsample with more upsample to class 4 which I visually noticed had more false classifications.
I equal ensembled the models with 5 TTA each with changing the aug. to just resizing image not cropping do have a different image view contrary the other 3 ensembles and skipped all other augs. just random brightness, contrast etc.
2 - Pytorch - 13 models
8 tf_efficientnet_b1_ns
5 resnext50-32x4d
Common for all trainings
Extra training data
BiTemperedLoss with Random Noise settings described in the
[https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html](https://ai.googleblog.com/2019/08/bi-tempered-logistic-loss-for-training.html)
but I used more narrow approach with 0.5/1.5.
OneCycleLRWithWarmup from Catalyst which also takes in count momentum and weight_decay in the cycles.
Used a random mix with almost equal split of all different aug. techniques SnapMix, Fmix, CutMix,CutOut, Mixup and MWH(Mixup-Without-Hesitation).
AdamP optimizer
[https://github.com/clovaai/AdamP](https://github.com/clovaai/AdamP)
Here I instead used  stacking with 5 TTA instead of ensemble to create a different approach to the global ensemble.
For the last 2 local ensembles I used 2 public trained models 5 B4ns and 5 resnext50-32x4d with 4 TTA for each model to have bigger ensemble and trainings to not get stuck in one’s own strategy.
[https://www.kaggle.com/piantic/cassava-resnext50-32x4d-weights](https://www.kaggle.com/piantic/cassava-resnext50-32x4d-weights)
[https://www.kaggle.com/underwearfitting/moa-b4-baseline](https://www.kaggle.com/underwearfitting/moa-b4-baseline)
Thanks, and credit to your training and models.
All 4 local ensemble where then ensemble with equal weights.
That’s it, thanks for reading the write-up.
This gave my 3rd silver medal, hard work, lots of reading and testing pay off.
I have also made a personal competition and challenge this time, to contribute in 5 different Kaggle competition in parallel, trying to get good results in every one of them, equal distribution of time. Just for fun and the learning. 1st out finished yesterday and got a bronze medal and now silver in this one, three to go, and hopes for the best, the learning is a big part of the value.
Happy Kaggleing!