# 9th Place solution

Thanks to all the organizers for hosting such a challenging competition. Also, thanks to all other competitors for sharing valuable information in forums and notebooks. Special thanks to
[@vlomme](https://www.kaggle.com/vlomme)
for sharing his code on github from last competition.
Prayers for all the folks and their families suffering from Covid.
Solution Summary:
For most of my models, I used 5-7 second clips for training. I gave lower weight to secondary predictions and used modified mixup just likes last year's 1st and 3rd place solutions.
Ensembling
Mostly, the strategy was to get as much diversity possible into the ensemble to make predictions more robust.
Melspecs with different temporal resolution (hop_length - 200 and 320)
resnest50, efficientnetb0 and densenet121 backbones
Noise augmentation (white noise, pink noise, band noise, nocall clips) same as what
[@vlomme](https://www.kaggle.com/vlomme)
used in previous competition
Post processing
This was the trickiest part. We had following information at hand in addition to raw probabilities for each 5s clip:
Maximum probability of bird over whole audio clip
Average probability of bird over whole audio clip
Count of birds from
train_metadata.csv
within +/- x deg of soundscape location
Maximum probability of bird over whole day
Initially I tried fitting a meta estimator (LightGBM and RandomForest, LogisticRegression) over above variable but results using custom rule were better on both validation and public LB (Thinking retrospectively, I think this is where I overfit).
Custom rule was something like:
p1 * a1 + np.clip(p2, 0, t1) * a2 + ...
Other tricks
Raise melspecs for different models to different powers to add little more diversity
Squeeze width of test soundscapes by 2-5% (mostly to reverse far field effects)
Things that didn't work
Using last name from common name (e.g.
parakeet
from
Orange billed parakeet
) as an additional label during training.
Different Attentions on backbone feature map (I guess with just 5-7 s clips, adding attention confuses the model even more instead of helping. Simple sum/mean over seems to work best.)
Bigger models, I had really hard time getting effcientnet-b2/b4 to converge. I just gave up on them due to lack on time and resources.
Manual labelling and class balancing. Looking at results from my models, it was quite evident that birds with less no of samples were not getting predicted (e.g.
hofwoo1
). So, I split them into 10s clips  and hand labelled them to increase their sample size but this didnt help. Also, class balancing gave lower validation loss to ditched it as well.