# 11st solution

I'm very grateful to the organizers. This competition was very challenging for me, and due to good fortune, we secured the last gold medal position.
Let me briefly report on our plan, no big difference between here:
[https://www.kaggle.com/competitions/birdclef-2024/discussion/497539](https://www.kaggle.com/competitions/birdclef-2024/discussion/497539)
Data
BirdCLEF2021, 2022, 2023, 2024 (this competition), ff1010bird_nocall, we didn't adopt additional mp3. When it was introduced, the LB had dropped, even though I found it odd.
Model
We adopted the model that ranked 4th last year, but swapped the backbone with efficientnet b0/b1/b2.
Feature
Log mel spec with:
= 
 = 
 = 
 = 
 = 
 = 
 =
Data augmentation
heavy augmentation improve my CV but LB drop.
A.HorizontalFlip(=0.5),
A.OneOf([
                    A.Cutout(=5, =16),
                    A.CoarseDropout(=4),
], =0.5),
and mixup v2:
MixupV2(nn.Module):
     __init__(self, mix_range=(., .), add_label=True):
        (MixupV2, self).__init__()
        .distribution = torch.distributions.Uniform(low=mix_range[],
                                                        =mix_range[])
        .add_label = add_label

     forward(self, X, Y, weight=None, teacher_preds=None):

         = X.shape[]
         = len(X.shape)
         = torch.randperm(bs)
         = self.distribution.rsample(torch.Size((bs,))).to(X.device)

         n_dims == :
             = coeffs.view(-, ) * X + ( - coeffs.view(-, )) * X[perm]
         n_dims == :
             = coeffs.view(-, , ) * X + ( - coeffs.view(-, , )) * X[perm]
        :
             = coeffs.view(-, , , ) * X + ( - coeffs.view(-, , , )) * X[perm]

         self.add_label:
             = Y + Y[perm]
             = torch.clamp(Y, , .)
        :
             = coeffs.view(-, ) * Y + ( - coeffs.view(-, )) * Y[perm]

         weight is None:
             X, Y
        :
             = coeffs.view(-) * weight + ( - coeffs.view(-)) * weight[perm]
             self.add_label:
                 = teacher_preds + teacher_preds[perm]
                 = torch.clamp(teacher_preds, , .)
            :
                 = coeffs.view(-, ) * teacher_preds + ( - coeffs.view(-, )) * teacher_preds[perm]
             X, Y, weight, teacher_preds
Training
First, we pre-trained the first 15 seconds on BirdCLEF2021, 2022, 2023 but removed the classes that were included in the 2024 data to avoid leakage. Then, the first 5 seconds of the 2024 data were used to train the final model. This training plan is the most stable we have found.
Postprocessing
Slightly improve the LB both Public and Private , My teammate will supplement further later.
Tried but not work
hubert and distill
distill or finetune birdnet
pseudo label
select multiple segs in an audio by model according to score (why? quite strange )
Notebook
Our single best model is a b2 based model, Public 0.68, Private 0.67.
We finally chose an ensemble of 6 models, Public 0.7, Private 0.67 without post processing
[https://www.kaggle.com/lihaoweicvch/lhw-final-en6-onnx](https://www.kaggle.com/lihaoweicvch/lhw-final-en6-onnx)