# Semi-supervised part of 20th solution

First I want to thank researchers at CVSSP, University of Surrey, UK for their great baseline
[code](https://github.com/qiuqiangkong/dcase2019_task2)
. My entire system is built upon it.
Pre-processing
Nothing special. 32 kHz sampling rate, 500 hop size so that 64 fps. Silence trimming below
max - 55 dB
, mel size 128. Random excerpt 4-second patches from entire spectrograms. Shorter spectrograms are padded repeatedly.
Data augmentation
1. Spec-augment without time warping
2. Time reversal with 80 more classes (binary target with 160 classes in total)
3. I'll talk about MixUp later
Semi-supervised learning
Almost the same as
[https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26](https://link.springer.com/chapter/10.1007/978-3-030-20873-8_26)
.
![](assets/d8c8aafe146777033c7983f8.png)
Suppose V X y_V is a batch of curated data and W is a batch of noisy data without label, then we have
![](
[assets/21faf27186f7a78206c4f9f7.png](assets/21faf27186f7a78206c4f9f7.png)
=400x320)
The backbone is 9-layer CNN, which is the same as the one in public kernel and CVSSP baseline.
Loss function
Focal + ArcFace.
ArcFace is modified for mult-label classification. 2 reasons to use ArcFace here
1. Enlarge the margin between positive and negative, so that the threshold eta becomes less sensitive.
2. In the
[original ArcFace paper](https://arxiv.org/abs/1801.07698)
, the authors state that ArcFace enforces less penalty towards samples whose feature vectors have large angle with the weight matrix than other margin-based loss. I suppose it would apply less "weight" to wrong pseudo-labels.
In practice using ArcFace makes my NN converge faster and have higher local lwlrap when using only curated data. I don't know what would happen if I trained the semi-supervised model without it because I started using it at a very early stage.
Training Details
First I warm  up the backbone CNN with noisy set. For the semi-supervised model, I have 6 samples from curated set and 58 samples from noisy set in each batch. Nadam optimizer, cosine annealing learning rate with linear warm-up are used. One can refer to my technical report, Table 2 for details about hyperparameters.
Inference
Arithmetic mean on 0.25s sliding window. TTA only on clips with short duration by repeating the sliding window with Spec-augment.
Single model trained on entire training set gives me 0.712 lwlrap on stage 1 test.
Again, thank the host for hosting such interesting competition. I am still on my way to my first gold medal :)