# 15th place solution

First off, I would like to thank Kaggle, the host, and my team
[@senkin13](/senkin13)
,
[@alexpartisan](/alexpartisan)
,
[@chenlongwang](/chenlongwang)
. It has been an interesting journey for me and certainly very rewarding to get my first gold. And Congrats to all who has worked hard for this competition!
I was mainly working on GNN while my teammates have worked on NN/lgb/feature engineering/stacking. So I will be focusing on GNN and leave the rest to them.
I mostly used
[pytorch_geometric](https://github.com/rusty1s/pytorch_geometric)
for GNN, which is an amazing library. Network structures I have tried but failed to work include NNConv (
[https://arxiv.org/abs/1704.02901](https://arxiv.org/abs/1704.02901)
), SchNet (
[https://arxiv.org/abs/1706.08566](https://arxiv.org/abs/1706.08566)
). I guess the reason is that those networks does not update edge information and this competition aims to predict edge attributes. I tried various ways (concat/add/multiply) to combine node information of the last layer into edge to make predictions but they do not work as well.
Our major break through is based on Graph Networks as a Universal Machine Learning Framework for Molecules and Crystals (
[assets/18db685182f3718128f9aa25.pdf](assets/18db685182f3718128f9aa25.pdf)
), in which both edge and node information are updated together. On top of the edge prediction, I add MLP per type to make the final prediction.
As for training, we tried both training each type separately and all types together and found that training all types together gives more stable results. We also tried fine-tune each type after training all types together first. But we found that it overfits very fast and leads to worse results.
One question I was never able to answer is that, there are two types of graph, one based on chemical bond and one based on coupling that we seek to predict. While they have the same node, edge connection is very different. Not having any background knowledge in this domain, we were not sure which one to train GNN on and left it to trial and error. Maybe someone with domain knowledge could shed some light on this.
Again, Congrats to all winners and look forward to their solution.
Best,
Zhenlan,