# 29th Discovery and Solution

I would like to express my sincere gratitude to Kaggle and University College London for hosting this competition. I learned a great deal, and it was truly an exciting and rewarding experience. I would also like to thank my teammates
[@ajinomoto132](https://www.kaggle.com/ajinomoto132)
[@chenzhenyuan](https://www.kaggle.com/chenzhenyuan)
[@atamazian](https://www.kaggle.com/atamazian)
[@larrylin666](https://www.kaggle.com/larrylin666)
—together, we proudly earned a silver medal in this competition.
Summary
Preprocessing:
Noise reduction
Stage 1:
Transit time prediction and smoothing of noisy samples
Stage 2:
Transit modeling and end-to-end neural network
Postprocessing:
Simple refinements for the transit model and ML-based sigma prediction for the neural network
Data Preparation
Based on the official submission baseline, we made the following modifications: we adopted a binning factor of 5 to preserve richer temporal information and applied background noise removal to improve signal quality.
augmentation
We augment the dataset by pairing each signal with all available calibration files, rather than only its nominal pair, to increase sample diversity and improve model robustness.
signal0 → sample1 
signal1 → sample2 
signal0 → sample3 
signal1 → sample4
smoothing and normalization
():
    
    batch_size, n_channels, n_wavelengths = train_signal.shape

     ():
        x = np.arange(-size// + , size// + )
        g = np.exp(-(x**) / (*sigma**))
         g / g.()
    gauss_coefs = gaussian_kernel(size=, sigma=)

    
    q = train_signal[:, :, -win:+win]  

    
    q = q / q.mean(axis=, keepdims=)

    
    q_coef = q.mean(axis=(,))  

    
    t_smooth = train_signal[:, :, -win:+win].copy()

    
     l  (win, t_smooth.shape[]-win):
        coefs = q_coef[l-win:l+win+] / q_coef[l]              
        window = train_signal[:, :, -win+l-win:-win+l+win+]  

        
        t_smooth[:, :, l] = np.tensordot(window * coefs, gauss_coefs, axes=([],[]))

    
     win > :
        t_smooth = t_smooth[:, :, win:-win][:, :, ::-]
    :
        t_smooth = t_smooth[:, :, ::-]

    
    first_col = train_signal[:, :, :]  
     np.concatenate([first_col, t_smooth], axis=)
Then we do normalization afterwards.
Adjust pipeline from physical principle:
Dark current represents an
additive noise component
in the measured signal. It should therefore be subtracted
before nonlinearity correction
, which is designed to operate on the
true physical signal
. Including dark current in the nonlinear transformation would violate this assumption and introduce systematic errors.
![formula](assets/77c977a688cf36c262f6493f.png)
Benefit:
Applying dark current subtraction before nonlinearity correction ensures that the polynomial correction operates on a cleaner signal, free from additive noise. This prevents the dark current component from being
amplified or distorted by the nonlinear transformation
, resulting in more accurate signal calibration and improved downstream performance.
modelling
Method 1 : NN with filter kernels, CNN backbone, and projection layer.
Method 2: transit model and tree model stacking
Visualization
Shown below are examples of typical anomalous samples frequently observed in the dataset.
distribution outliers
![distribution_outliers_examples](assets/64a0e477945a12f07dc67924.png)
extreme values
![extreme_values_examples](assets/e4ec69cc17b1147f6b546918.png)
gradient anomalies
![gradient_anomalies_examples](assets/17658c3536ae3336906b484d.png)
phase anomalies
![phase_anomalies_examples](assets/7c827006adfba3ed9c137001.png)
stability anomalies
![stability_anomalies_examples](assets/c75a89651dd5a2634e44d486.png)
transit anomalies
![transit_anomalies_examples](assets/1b56b9acc8d0bbf866714850.png)
Postprocessing
We didn't figure out robust postprocessing.
Conclusion
In this work, we built upon the official baseline by introducing a series of targeted improvements across data preprocessing, augmentation, modeling, and calibration. This competition reinforced the importance of careful data preprocessing, the combination of domain knowledge with machine learning, and iterative experimentation.
Cheers to all Kagglers, and really have fun!