# First time Competition, 75th place solution silver region

This competition is indeed coming to an end. With 3 days remaining(as this post was made), lets overview important things that I found quite useful learning wise. This competition is my first competition, and set up a really good impression of kaggle for me. I believe that this competition is a great competition to not only learn, but to test your knowledge and discover new ideas.
Notebooks down below.
With over 3.k+ members attending, 4.8k competitors, with over 85.5k+ submissions…
Cassava 2019 tournament: A classification problem of this tournament back in 2019. Winning solutions for this classification are
1st place:
[https://www.kaggle.com/c/cassava-disease/discussion/94114](https://www.kaggle.com/c/cassava-disease/discussion/94114)
2nd place:
[https://www.kaggle.com/c/cassava-disease/discussion/94112](https://www.kaggle.com/c/cassava-disease/discussion/94112)
3rd place:
[https://www.kaggle.com/c/cassava-disease/discussion/94102](https://www.kaggle.com/c/cassava-disease/discussion/94102)
These notebooks came in really handy, as they provide useful ideas that we competitors can integrate in.
Some useful notebooks in this competition that benefitted me and many others.
(Note: These are not all of the helpful notebooks that have helped me, but instead the notebooks I believe started me up the fastest.)
[Dimitre's cassava leaf disease training](https://www.kaggle.com/dimitreoliveira/cassava-leaf-disease-tpu-tensorflow-training)
[Heroseo's various loss functions](https://www.kaggle.com/piantic/train-cassava-starter-using-various-loss-funcs)
[Kun Hao Yeh's inferencing notebook +training](https://www.kaggle.com/khyeh0719/pytorch-efficientnet-baseline-inference-tta)
[Y.Nakama's notebooks](https://www.kaggle.com/yasufuminakama/cassava-resnext50-32x4d-starter-training)
Step 1: Knowledge distillation
My team and I realized that this dataset marginally have a lot of noisy labels, and so in order to address this problem.. We smoothed the labels and later combined knowledge distillation to feed into our model.
Step 2:
We realized that certain augmentations are really useful of course during training stage.
With 8x TTA in inference giving us a
Public LB 0.904
and
Private LB 0.900
While 5x TTA was a tad bit worse.
Step 3:
Center crop + hue saturation effected out PB LB, alongside with 5 folds for efficientnetB4.
Because the training time was 8 hours max, all my teammates trained separate folds for each and later did the same to resnext for inference. You can check this out on the inference and training notebook down below.
Efficientnet B4 + Resnext:
Our best fit model with no distillation was:
Model
Distillation
Public LB
Private LB
Efficientnet B4
No
0.899
-
Efficientnet B4
Yes
0.902~0.9003
0.9034
Efficientnet B4 + Resnext50
Yes
0.903~0.904
0.90036
Experimentations
Augs
:
Fmix
SnapMix
Training: Hue Saturation  + Coarse dropout + Random Brightness
Inference: Standard TTA augs
TTA:
Light augs(only flips): 4~8x TTA
Standard augs: 5~8x TTA
Losses
:
Taylor crossentropy + smoothening(best)
Bi-tempered loss
Categorical crossentropy
Conclusion
Lucky to not have a bad shakedown!
We did try to implement NFnet, however we didn't have time when the weights were released so we bailed on that. Check it out as it is revolutionary!
Again, these are just a list of the things that came off the top of my head! First competition, and I don't know if it is finalized so I won't be uploading our notebook at the moment for solution. However, I will be uploading it tomorrow! Just making sure that the place my amazing team
[@teodorbogoeski](https://www.kaggle.com/teodorbogoeski)
and
[@varungadre0910](https://www.kaggle.com/varungadre0910)
are finalized! Still don't quite understand how kaggle really works but definitely interested in learning more! If this is finalized, we will be placed 75th on the first competition that my team and I have done! Learned lots from kaggle and this competition! Definitely going to do more… Thanks to many wonderful people who shared their ideas and knowledge in a meaningful manner for novices like me.
(Edit): Never mind, turns out I will release the notebooks now.
I will be planning to upload notebooks for various datasets… Just waiting for this to finalize just to be sure! Thank you kaggle!
Training:
[Here](https://www.kaggle.com/andyjianzhou/cassava-training-fastai-efficnetnet-distill)
Inference:
[Here](https://www.kaggle.com/andyjianzhou/cassava-inferencing-efficientnetb4-resnext50?scriptVersionId=54404877)
Thanks to my amazing teammates who built these models efficiently! Looking forward to next competitions!
Useless note: Did pretty bad on my unit final today but that's alright.. Machine Learning Forever!