# Highest PublicLB Kernel posted (#29 Solution)

First of all I would like to thank Kaggle team and various data donators for this challenging data-set. Starter kernel is posted is
[here](https://www.kaggle.com/algorrt/highest-scoring-public-kernel-starter-sol-29?scriptVersionId=12845117)
I have mentioned scores wherever I have found some landmarks:
pubt : public train accuracy
prit : private train accuracy
What I tried
- First tried running with all parameters included just to get a baseline score (
pubt : lower 0.6x, prit : higher 0.5x
).
- Next I thought to create columns featuring differences in adjacent values. This showed some promising results also this features were having higher weightage predicted by lgbm. However, I was still having some doubts related to orientation values (
pubt : lower 0.6x, prit : higher 0.5x
).
- Removed all orientation columns and reran, Values fell straight, but now it appears it has better private score (
pubt : higher 0.4x, prit : lower 0.6x
).
- Later, I tried to predict robot's velocity with various factors present but had to give this due to 
unsatisfying results.
- Meanwhile,
[Christoffer Karlson](https://www.kaggle.com/christoffer)
frequency] established
[Sampling frequency](https://www.kaggle.com/christoffer/establishing-sampling-frequency)
. This turned my attention to frequency domain and trying to use fft's here.
-  As I was still testing,
[Thomas Rowher](https://www.kaggle.com/trohwer64)
shared this awesome
[FFT Analysis](https://www.kaggle.com/trohwer64/simple-fourier-analysis)
and shared his insights
[here](https://www.kaggle.com/c/career-con-2019/discussion/87239#503272)
.
- Also, I was smoothing the values present in signal, with
[Savitzky-Golay](https://en.wikipedia.org/wiki/Savitzky%E2%80%93Golay_filter)
filter rather than averaging them out. This is great to increase SNR without distorting signal too much.
- I started to work heavily with fft's(peaks, range, power) and I found some improvements but still I felt something was missing (
pubt : lower 0.7x, prit : higher 0.6x
).
- I was thinking if maybe we could have more values in every set, and
[Markus F](https://www.kaggle.com/friedchips)
had an amazing find -
[Links between sets](https://www.kaggle.com/friedchips/the-missing-link)
. 
- Next, I tried to use more samples in every set, however I think maybe I miscalculated and had some error which resulted in lower scores. I gave up this idea.
- Finally, I linked all the test set & training sets for fun. Thought what we can see if we linked entire test with train. 
- I found a total of 151 linked sets. In which, 104 sets contained some data from train sets. These were somewhat easy to predict or so I thought.  Remaining 47 test sets were not linked to any train data. Here I put in data which I modeled with RF. And with this public train score raised to 0.96. It appeared as if I had hit gold (
pubt: 0.96, prit: 0.68
). I will soon publish a kernel which uses Markus links along with RF model which gives this score. Posted, check link above :)
- However, I had a feeling this was somehow overfitting the train and public test sets. It was!!
- I then tried to check for all the sets where different surfaces got linked. Once plotted, it was somewhat clear which were wrongly linked. Did some post-processing and reran(
pubt: 0.98, prit: 0.76
).
This is the first competition I have participated strongly and it was great to learn from many people here. I would like to specially thank :
[Thomas Rohwer](https://www.kaggle.com/trohwer64)
[Markus F](https://www.kaggle.com/friedchips)
[Christoffer Karlsson](https://www.kaggle.com/christoffer)
[Gilles Vandewiele](https://www.kaggle.com/group16)
[Nanashi](https://www.kaggle.com/jesucristo)
Looking forward for more solutions from everyone :)
PS. The idea underneath this kernel was floating on forum for more than 2 days before the competition end.