# Notes on G2 Comp

In total, I trained 96 full-fold models for this competition. Having nice local compute helps, but having deep insight into the problem + good exploratory prowess is more important (both of which I lack). I guess predestination / luck also plays its role :-). Some of the things I had tried off the top of my head (since I've already deleted my working directory):
Channel shuffle
Channel shifting
Random noise
1D Net, all my 1D nets were really shallow, just 4 layers w/o residual
2D Net, couldn't get them to converge as nicely as the rest of folks
Downloading actual LIGO 3 detector data for pretraining - it's very messy and has bands not represented in the synthetic data
GW injection using pycbc in 1d and 2d
Injected gw param regression as pretrain task
Cutout
Mixup
Trainable CQT
Very wide kernel sizes
Kernel smoothing and resume training
Frequency domain CQT
FD 1D net, 6 channels
Training on bandpassed first derivative data
Training on bandpassed integrated data
PCA, SVD, and FastICA transformations as channel augmentations
Signal flipping (only works in 1D
x=-x
. You can train a model like this then do TTA)
Signal compression and stretching - I only attempted this in 1D as TTA and for training
Random Mean and STD shift TTA
Whitening by creating ~6sec signals: basically I'd concat the signal with itself on the right and left, but first I'd find the location on the right that has the most similar value and slope and repeat from there. Same as the left. I thought by using longer signal, we take care of the ASD generation issue. It 'worked' but the model did not benefit
Whitening using auto regression residuals
Swin Transformer
Galerkin transformer in 1D time domain, bucketing signal into 8hz wide chunks
Probably more that I am forgetting
I feel like instead of trying 903209230923 things, better to abandon quickly what doesn't work, focus on what does, and importantly, drive it to its logical conclusion. I was very early on the 1D train and was training nets with the full competition data in ram. But after I got to a decent level in 1D (at the time) I started exploring all these different avenues instead without focusing more in improving the net's architecture.
I'd also like to express gratitude and appreciation to my team mates. I actually ran out of steam about two weeks ago, but contributions and motivation from
[@misaki1111](https://www.kaggle.com/misaki1111)
and
[@jpison](https://www.kaggle.com/jpison)
helped carry me over. Thank you.