# A solution and some learnings

First I want to congratulate the winners and thank the organizers for a fantastic competition. It is great to have the scientific background of the competition. I feel like I just started and hope there will be a continuation.
I have read through the descriptions of the top solution. There were parts that I had included, others I had worked on but not managed to complete, as well as ideas I had not thought of. I would like to give a (hopefully) short description of the solution and what was missing.
Data normalization. 
For data normalization I used features luminosity = flux*distmod**2 and magnitude = -2.5 log10(flux) – distmod (+ 27) for extra galactic objects. I did not use the normalization as in
[https://www.kaggle.com/c/PLAsTiCC-2018/discussion/75054](https://www.kaggle.com/c/PLAsTiCC-2018/discussion/75054)
I would like to try that out.
Initial Models.
I split the models early into galactic and extra galactic. The initial models were NN, catboost, lgb and xbg models. 
For NN I started with the model given by
[https://www.kaggle.com/meaninglesslives/simple-neural-net-for-time-series-classification](https://www.kaggle.com/meaninglesslives/simple-neural-net-for-time-series-classification)
but changed the features to luminosity and added 1-3 layers with a tower of 3 CNNs(64/32/16), Dropout and MaxPooling/AveragePooling for processing a group of features for the 6 bands. 
This was followed by a sequence of Dense layers. This was a simple NN that gave good initial results. I was not able to improve this essentially, but used it as input for stacking and averaging. 
I used catboost, lgb and xgb with depth 3, initially with 5 folds and later with 10 folds. The best results were initially with catboost followed by lgb, but xgb results were worse. Of course that required much feature engineering and took some time to pass the simple NN. When using NN as input for stacking the results of lgb became best and xgb were good enough to use in weighted averaging. 
This was then the final Architecture with base models with NNs, catboost and lbg, and lgb and xgb models using stacking with NN and catboost results as input. This was then followed by weighted averaging including the base models.
Feature Engineering and PCA
There are extremely many possibilities of creating features. The most difficult ones are for differentiating the hard classes that seem to be different types of supernovae. For this I created, as many others
•   Features restricted to some areas: detected area between first and last detected = 1; area where magnitude is below a limit; a growing/falling area prior/after maximal flux but within detected area, dividing by maximum to use ratios, and including counts to help the models
•   Taking differences between steps and divide by mjd difference to have derivatives and statistics based on the derivatives. This is to get the slope during the growth and falling of luminosity as well as the changes in the slope.
•   Taking average between steps and multiply by mjd difference for summing up to get integrals of the luminosity over the growing and falling area. This was followed by normalization to get a description of the shape.
•   Additional features such as A, eta described in
[assets/496de40c9b8df5bc4f1dfde6.pdf](assets/496de40c9b8df5bc4f1dfde6.pdf)
•   autocorrelation__lag_1, fft_coefficients, entropy, skewness (but never kurtosis) (using tsfresh) and stetson_k
When adding such features to models this often lead to worse results. For handling that I included the use of PCA on the 6 bands to reduce the dimensionality of such features from 6 to a lower dimension like 1, 2 or 3.  This made it possible to use much more features and to use them together. I have not seen this use of PCA mentioned here before.
What failed or could have been better
The part that I worked on and did not manage to complete successfully was curve fitting and use of Bayesian techniques. For curve fit I used Bazin. That gave extremely good results in CV, but the initial features gave bad results on LB. I was not able to complete this together with Baysian techniques in time. The main difficulty was the difference between train and test set. Here I did not manage to setup a reasonable process including augmentation. This was a great part of the solution by Kyle Boone. I think it is important to identify such essentials early enough to focus on the right parts.  That will be my main follow-ups including complete the missing parts of the solution (and get experience in using Gaussian Processes).