# 37th place solution

First of all, I would like to thank the hosts for the preparations, my teammates
[@imayushsaxena](https://www.kaggle.com/imayushsaxena)
and
[@octaviograu](https://www.kaggle.com/octaviograu)
and all the competitors who made it that challenging!
Our top solutions
The solution got
variant
public
private
LB place
Final place
10th
0.6152848
0.6198108
53
-
11th
0.6128403
0.6293136
-
37
Summary of 11th
We used only AIRS data for the solution
The basic flow of the solution:
Standard correction, but without binning / shrinking the flux
Normalize the signal
Denoise the signal with
gaussian_filter1d
Detect transit phases, including limb darkening
Calculate features for the whole signal
Calculate features for binned wavelengths by grouping nearest 6, 10, and 25 wavelengths.
On top of the features 1000 models were trained
The mean of the model's predictions gives the transit depth prediction
Sigma is based on the std for the model's  predictions per wavelength
[Link](https://www.kaggle.com/code/thegrey/ad24-eleventh?scriptVersionId=204479373)
to the 11th solution.
Backbone pipeline
We tried many denoising strategies, so we needed a simple way to iterate. We built a functional pipeline that's easy to configure:
functools  reduce

full_pipeline = [
      **kwargs: bin_wavelengths(**kwargs),
      **kwargs: normalize_signal(**kwargs),
     **kwargs: filter_signal_with_gaussian(sigma=, **kwargs),
     **kwargs: polynomial_detrend(**kwargs),
     **kwargs: fit_transit_poly(**kwargs),
     **kwargs: collect_features(**kwargs),
]

initial_kwargs = {: , : , :, :}
result = reduce( kw, func: func(**kw), full_pipeline, initial_kwargs)
Then we can combine
result['features']
for different pipelines.
Each step in the pipeline can be organized like this:
():
    signal = kwargs[]
    filtered_signal = median_filter(signal, size=window)
    kwargs[] = filtered_signal
     kwargs
Transit phase detection
After experimenting with the
[BATMAN](https://lkreidberg.github.io/batman/docs/html/index.html)
library, we found that there are several possible structures of the transit flux signal, even when fully denoised and detrended. Limb darkening can influence the calculation of transit depth, affecting how we calculate the transit edges
():
    signal = kwargs[] + 

    grad1 = np.gradient(signal)
    mn = np.argmin(grad1)
    mx = np.argmax(grad1)

    peaks = find_peaks(grad1, width=)[]
    ingress_left = peaks[(peaks < mn) & (mn - peaks <= )][-]  np.((peaks < mn) & (mn - peaks <= ))  mn-
    ingress_right = peaks[(peaks > mn) & (peaks - mn <= )][]  np.((peaks > mn) & (peaks - mn <= ))  mn+

    dips = find_peaks(-grad1, width=)[]
    egress_left = dips[(dips < mx) & (mx - dips <= )][-]  np.((dips < mx) & (mx - dips <= ))  mx-
    egress_right = dips[(dips > mx) & (dips - mx <= )][]  np.((dips > mx) & (dips - mx <= ))  mx+

    kwargs[] = ingress_left
    kwargs[] = ingress_right
    kwargs[] = egress_left
    kwargs[] = egress_right

     kwargs
Features calculations
Knowing the
ingress
and
egress
of the transit, we fit a polynomial into the flux signal (ignoring the transit) and then perform detrending.
Then we can assume that once the plain detrended signal is on the level of constant 1, then we can derive a transit depth by the formula:
():
     /s -
Here
s
can be a mean flux in the transit, flux at the middle point, etc. We collect quite a few here. This was a huge speed improvement compared to the
minimize
method.
Additionally, we collect generic features on the in-transit flux, like the standard deviation.
Models training
We have generated 500 Ridge models and 500 PLSRegression models that formed the ensemble.
Alpha for Ridge was random:
alpha =  ** ( -  * np.random.rand())
The same as
n_components
for PLSRegression:
model = PLSRegression(n_components=n_components, tol=)
We used different subsets of the training data for each model:
subset_size = (X_train.shape[] * np.random.uniform(, ))
We sorted the model predictions for each wavelength, removed the lowest and highest 30%, and took the mean of the remaining predictions.
Sigma calculation
We calculated the standard deviation of the sorted predictions for each wavelength:
std_preds = np.std(sorted_preds, axis=)
We then multiplied the standard deviation by a coefficient that worked best in LB:
COEFF_0 =  
sigma_response = sigma * COEFF_0
What worked well, though didn't make it to the final submission
Usage of wavelets (
pywt
) - our favorite one was
db29
median_filter
savgol_filter
What didn't work
Gaussian Process Regression - we tried different kernels here, but didn't see breakthroughs while a big increase of the processing time
LinearRegresion, SVM, LGBM, CatBoost, RandomForest, XGBoost…
BayesianRidge - worked well, though long time (and didn't fit into the memory for more features)
We tried to train a neural network with a loss function for both sigma and transit depth (the competition score), but didn't have much luck due to the limited time
Improvements We Lacked Time For
Generate the train data with BATMAN
Use FGS data
Use data for real molecules to generate additional training data
Collect more meaningful features (and filter out those which don't help)
Sigma approaches - we didn't use all the potential here
Please feel free to explore our
[best solution](https://www.kaggle.com/code/thegrey/ad24-eleventh?scriptVersionId=204479373)
.