# 65th place writeup - bronze

This was an awesome competition and I learned a lot from it. This is due to the hosts, to my teammates, and the kagglers who shared ideas!
A few things about our solution:
Ensemble of SED models with two different backbones: seresnext26tn_32x4d and  tf_efficientnet_b0_ns, both from timm, 5 folds each, 30 epochs each run
Each backbone used a different seed
Both backbones used primary and secondary labels in training
Everything was trained on google colab notebooks
The optimal threshold for the ensemble was 0.25
Audio augmentations and image augmentations
Cutmix and mixup
CosineAnnealingLR (Tmax=7 for seresnext and Tmax=4 for effnet). Couldn't find any improvements using other schedulers
seresnext was trained with the first 5 seconds of each audio file
tf_efficientnet_b0_ns was trained with seconds 5 to 10 when audio file has more than 10 seconds and with first 5 seconds otherwise
We used
[focal loss](assets/1a7d7f75842f50d9dcb4a05e.pdf)
, tuned its parameters, and found that using alpha = 0.75 and gamma = 5 gave a huge boost on our models
From the moment we started using gamma = 5 in focal loss we had good CV/LB correlation for most cases, using
[@dschettler8845](https://www.kaggle.com/dschettler8845)
[method](https://www.kaggle.com/competitions/birdclef-2022/discussion/314999)
We also monitored another computation of f1 score, similar to the one above but only for scored birds. So we could track the learning of scored birds, which was slower than the other birds, but in the end of training both CV strategies and calculated f1 were quite similar
Things that didn't work:
Noise reduction
PCEN (Per-Channel Energy Normalization)
[GeM](https://arxiv.org/abs/1711.02512)
AdamW
Pseudo labels via oof predictions (we tried this a lot but couldn't make it work)
Reducing dataset size or selecting samples from it
Again, thanks to my teammates who contributed a lot to this score:
[@gabrielvinicius](https://www.kaggle.com/gabrielvinicius)
[@lucasdmr](https://www.kaggle.com/lucasdmr)
[@paulojunqueira](https://www.kaggle.com/paulojunqueira)
[@felipemandrade](https://www.kaggle.com/felipemandrade)
We put a lot of time, effort and dedication to achieve this result.
In my opinion, what made this problem really hard was the weak labels. We tried to deal with this using pseudo-labels but we weren't successful on this.