# A technique that worked for me (~0.001-0.002 improvement) that none of the posted (so far) solution used

That is making sure that each batch has some high and low SNR sample (I designed each batch so that 30% of each batch has low SNR). This help preventing bad gradients from overfitting the solution.
Since the host didn't include other parameters and SNR information, I decided to use a simple model's output as an indicator of SNR. Basically just get any model that has at least 0.87 AUC, get its prediction on training model, and then decide a cutoff confidence score and compute abs(pred-label). SNR = high if abs(pred-label) < cutoff else low (I used 0.5 as my cutoff).
This technique improved my 1D from 0.8738 to 0.8760 and my B0 from 0.8749 to 0.8757. Although curiously enough, it degraded my best single model B3 of 0.8783 to 0.8777. That said, I discovered this like 2 days ago so I didn't have time to tune it properly.
P.S. If you train your model on decreasing order of SNR, your model will overfit to oblivion within the first epoch.
P.S.S. Can anyone point me toward the notebook that
[https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/275316](https://www.kaggle.com/c/g2net-gravitational-wave-detection/discussion/275316)
is complaining about?