# #57 Summary

This turned out to be one of my favorite contests. I have always been fascinated by chemistry and have a reasonable knowledge base in chemistry in general. Originally, I hoped to use my chemical intuition to get an edge in this contest. Bu I realized soon enough that, since I had zero knowledge in molecular modeling, my edge was very slim.
I sat on the fence for a very long time reluctant to invest time in learning the state of the art of molecule representation and modeling of molecular properties. Luckily, the Kaggle community came through with many great kernels that I could sample quickly and zero down to a few that were most promising in my hands.
I started working hard on this contest about two weeks ago. For a long time I was out of medal range. Then, things started to click around one week ago and I managed quickly move into solid silver range within a few days.
There was not much time left and models took long time to run. Since there was very little danger of overfitting to LB from what I observed on oof predictions, I chose not waste resources on creating oof prediction and blend old models with newer versions as they came in using LB as the only guide. The current standings and scores: #57 and -2.21819 (Private) and #58 and -2.22542 (Public), prove that that was a solid strategic choice.
The three kernels below (with minor tweaks) contributed the most to the final blend:
[SchNet Starter Kit](https://www.kaggle.com/toshik/schnet-starter-kit)
by
[@toshik](/toshik)
, who barely missed Gold with -2.8 solo score! Amazing!
[Distance - is all you need. LB -1.481](https://www.kaggle.com/criskiev/distance-is-all-you-need-lb-1-481)
by
[@criskiev](/criskiev)
, whose team got Silver!
[Keras Neural Net and Distance Features](https://www.kaggle.com/xwxw2929/keras-neural-net-and-distance-features)
by
[@xwxw2929](/xwxw2929)
, who got solo Bronze!
Basically, most credit for my 🥈  goes to the creators of these great kernels. I am glad they all did great in the contest. Congrats!
At first the core of my blend was Sergii’s kernel, then Aicanghai_Smile’s Keras version started to weigh in more, but, in the last 5 days or so, I realized that
the biggest hidden gem is actually SchNet.
Wow! In the final blend it has by far the most weight.
For SchNet my main modifications was training separate models for each type and adjustment of batch size upward (often I used 16 for mini-batches). Each of 8 models took 6-8 hrs to train for 60-90 epochs using Kaggle’s GPUs. The best SchNet model was -1.971 and I feel that I barely scratched the surface with what it could really accomplish.
99% of computing was done with Kernels. It is great to have access to such powerful GPUs. My main wish is that GPU kernels would not freeze two days before the deadline.
In the end, I ran out of time before I could apply my secret chemical intuition skills : (.  I am guessing that a promising strategy to improve further is to split molecules into finer subgroups based on molecular features such as double bonds, rings, etc. and then train separate SchNet models on these subgroups.
Congrats to all! Thank you the organizers and Kaggle for making this a great contest!
**
Update: *
[SchNet Kernel by J-Type](https://www.kaggle.com/stocks/schnet-by-j-type-helped-me-win-silver-57)