# Learnings From the Competition ,  39th Place Solution

Hi all,
I really wish I could have started this competition sooner but I did the best I could in 10 days . It never feels good to go down in private even if its 1 place , but what really was more disappointing was that we failed to pick our best submission which was giving
0.66
on private. It was an absolute grind for us , tons of hard work
Our bird didn't fly but it was a great experience .
I would like to thank
[@nischaydnk](https://www.kaggle.com/nischaydnk)
for agreeing to join this competition this late and putting in immense work , an absolute gem to have as a teammate
and my other teammate
[@parthdhameliya77](https://www.kaggle.com/parthdhameliya77)
,
[@vatsal](https://www.kaggle.com/vatsal)
and
[@drthrevan](https://www.kaggle.com/drthrevan)
for accepting us to the team
It was my first serious experience with audio data and
[this](https://www.youtube.com/watch?v=iCwMQJnKk2c&list=PL-wATfeyAMNqIee7cH3q1bh4QJFAaeNv0)
youtube playlist helped me get up to speed and really get the hang of audio data. Thanks to kaggle community and winners of the previous Birdcall competition sharing their solutions , it was easy for us to get the baselines up and ready
Things that worked
Using Delta for stacking instead of stacking the same spectograms 3 times ( Idea taken from Birdcall's 4th place solution )
RMS for cropping audio instead of random cropping  ( Idea taken from Birdcall's 4th place solution)
Resnext and Nfnet series of models
Post Processing
Things That didn't work
I tried to mimic last year's 1st place solution SED model
[here](https://www.kaggle.com/tanulsingh077/sed-model-tanul-s-version)
but the results weren't great because one epoch was an hour long and it took atleast 35 epochs to converge , we were outdone by the limited hardware power and time here . But we learned the in and out absolute working of the model
CPMP's sampling method : Instead of taking random crops , crop from the begining and the end
Efficientnet Family of models , I tried to mimic last year's fourth place training strategy but it didn't work out
Longer time durations (10 secs , 15 secs melspecs) , different temporal dimensions (changing hop size and frame size)
Vision transformer models , we used code provided by the organizers to generate a square mel specs but that mess up the temporal dimension I guess and hence the results were not that good
Things that we missed
We used already saved mel spectograms to train our models as we were limited by hardware and time and hence we were not able to use any of the audio level transforms and now I feel we missed the most there .
The key was to make the model robust to noise and overfit less
Failing to use the meta-information given to us in terms of location and time
Our Final Solution Summary
Our final model consisted of Nfnet_l0 (5folds) and 2 resnext50 (5folds) (with and without delta stacking)trained on 7 secs mel specs ,sigmoid level ensemble with a post-processing layer . The post-processing was something which gave us the biggest boost , it was
[@nischaydnk](https://www.kaggle.com/nischaydnk)
great idea , so I will let him explain it in the comments.
I take away a lot of learnings from this competition and I will be waiting eagerly for the next audio competition