# 14th place solution (GCN from scratch in Kaggle kernel)

For this competition, I implemented a GCN using keras.
Input features
:
1/ Atom types (one-hot encoding)
2/ Bond adjacency
3/ Bond types (one-hot encoding, generated using Open Babel)
4/ Bond vectors, normalized
5/ Inverse distance between all atom pairs, together with powers up to 4
6/ Cosine of bond angles for pairs of atoms 2 apart (for which we have to predict the coupling)
7/ Cosine of dihedral angles for pairs of atoms 3 apart (for which we have to predict the coupling)
8/ A flag on atoms whose dihedral contribution come from more than one path because they are part of a loop (in which case the max dihedral angle is retained, which is probably suboptimal)
Most features were created "offline", and pickled in a compact form to avoid exhausting the Kaggle instance memory. The data is "decompressed'' per-batch by a keras generator object. All the information regarding pairs of atoms are input in a 29x29 tensor, where 29 is the maximum number of atom per molecule. Entries for non-existing atoms are just zero'd, and masks are used where necessary.
Data augmentation
:
Random rotations to the bond vectors, to enforce independence from the reference frame.
Network structure
:
The network consists of 4 sections.
1/ The vertex (atoms) and edge (atoms x atoms) information is processed by 4 layers of width 1 convolutions (i.e., the same fully connected network is apply to each vertex independently, and similarly for the edges).
2/ 6 graph convolution layers are applied (transforming the vertex data, but leaving the edge data constant).
3/ A special layer passes the information from the vertices to the edges.
4/ 4 layers of width 1 convolutions are applied to the edges, to produce the magnetic coupling predictions.
Originally I only had 2 (body), 3 (head) and 4 (tentacles) :)
Extra data
:
The extra training data not present on the test data was used as follows.
1/ Mulliken charges, and shieleding tensor eigenvalues were predicted by two width 1 convo from the output of the the 3rd GC layer, and the prediction input into the 4th GC layer.
2/ The magnetic contributions are predicted all together at the output of the network and summed to provide the final predictions.
Unfortunately, only the eigenvalues of the shielding tensor can be used because the tensor are not provided in the correct reference frame.
Regularization
:
All layers are followed by ReLU nonlinearities, and most preceded by a Batch Normalization layer.
Result
:
This produces a score of -2.6 after about 27 hours runtime of the script (kernel) (3 times 9 hours continuous runs, with weights saved in-between). Averaging the predictions from about 10 runs yield the -2.9 final score.
Potential improvement
:
I was working on implementing a GCN with operations equivariant to rotations (as in
[https://arxiv.org/abs/1802.08219](https://arxiv.org/abs/1802.08219)
and
[https://www.kaggle.com/c/champs-scalar-coupling/discussion/106275#](https://www.kaggle.com/c/champs-scalar-coupling/discussion/106275#)
), but didn't have enough time to finish.
Script
:
[https://www.kaggle.com/cedben/squid?scriptVersionId=19764703](https://www.kaggle.com/cedben/squid?scriptVersionId=19764703)
[edited: network description]