First of all, I want to thank everyone who participated in this competition and the competition organizers.
\n\nThis was only my 2nd competition and I am really happy to have achieved my first ever medal and my first ever silver medal!
\n\nOverview\n- Every coupling was treated as its own graph\n- For the same molecule, graphs of 2 different couplings were different from each other.\n- Used the MPNN from the Gilmer paper https://arxiv.org/abs/1704.01212\n- Used basic chemical features like atomic number and basic geometric features like angles and distances.\n- Had same features for all types but different connectivity for 1JHX, 2JHX and 3JHX\n- Most important part was not the model but how the molecular graph was connected together \n- All geometric features were relative to the atoms at atom index 0 and 1 and 1 or 2 other atoms which I found.
\n\nMolecular Graph Representation\nIn the Gilmer Paper, a molecule is represented as a fully connected graph i.e. there are the default bonds (real bonds) and on top of that each atom is connected to each atom through a fake bond. In the paper, the point is to predict properties that belong to the whole graph and not to a particular edge or a node. So, in order to adapt to the nature of this competition, I used the following representation:
\n\nType 1JHX\n- Connected each atom to the 2 target atoms (atom index 0 and 1) on top of the default real bonds (note how this is not the same as the Gilmer paper where the graph is fully connected)\n- All geometric features were calculated as relative to the 2 target atoms.
\n\nType 2JHX\n- Found the atom on the shortest path between the 2 target atoms. So there were now 3 target atoms (atom index 0, atom index 1, atom on shortest path)\n- Connected each atom to the 3 target atoms on top of the default real bonds.\n- Features were calculated relative to all 3 target atoms e.g. distance & angle to atom index 0, atom index 1 and the atom on shortest path.
\n\nType 3JHX\n- Found the 2 atoms on the shortest path between the 2 target atoms. So there were now 4 target atoms (atom index 0, atom index 1, 1st atom on shortest path, 2nd atom on shortest path)\n- Connected each atom to the 4 target atoms on top of the default real bonds.\n- Features were calculated relative to all 4 target atoms.
\n\nAlso, I made all the graphs fully bidirectional. Using a fully bidirectional graph gave me a significant improvement over a one-directional graph which was used in the paper.
\n\nModel\n- The model was really basic with some additional layers and slightly larger dimensions, very similar to what is written here https://github.com/rusty1s/pytorch_geometric/blob/master/examples/qm9_nn_conv.py.\n- I added very little Dropout and BatchNorm in the initial linear transformation layer which actually led to the model performing better. \n- I experimented with adding Dropout in the MLP used by the NNConv and it showed promising results but they were too unstable so I decided to not go through with it.\n- I tried adding an attention mechanism over the messages passed by the network but did not see an improvement in score (most likely implemented it incorrectly)\n- I also tried using the node vectors of the target atoms only to predict the scc but this actually performed way worse (probably because the way I am representing my molecules does not translate well to using just the node vectors of a subset of nodes)\n- I only trained a single model for each type (8 models total) so did not do any ensembling
\n\nTrain only data\nUnfortunately, towards the end of the competition I was busy with some other work so could not get a chance to play around the fc, pso etc features.
", "messages": [ { "id": "613685", "postDate": "08/30/2019 18:23:29", "content": "First of all, I want to thank everyone who participated in this competition and the competition organizers.
\n\nThis was only my 2nd competition and I am really happy to have achieved my first ever medal and my first ever silver medal!
\n\nOverview\n- Every coupling was treated as its own graph\n- For the same molecule, graphs of 2 different couplings were different from each other.\n- Used the MPNN from the Gilmer paper https://arxiv.org/abs/1704.01212\n- Used basic chemical features like atomic number and basic geometric features like angles and distances.\n- Had same features for all types but different connectivity for 1JHX, 2JHX and 3JHX\n- Most important part was not the model but how the molecular graph was connected together \n- All geometric features were relative to the atoms at atom index 0 and 1 and 1 or 2 other atoms which I found.
\n\nMolecular Graph Representation\nIn the Gilmer Paper, a molecule is represented as a fully connected graph i.e. there are the default bonds (real bonds) and on top of that each atom is connected to each atom through a fake bond. In the paper, the point is to predict properties that belong to the whole graph and not to a particular edge or a node. So, in order to adapt to the nature of this competition, I used the following representation:
\n\nType 1JHX\n- Connected each atom to the 2 target atoms (atom index 0 and 1) on top of the default real bonds (note how this is not the same as the Gilmer paper where the graph is fully connected)\n- All geometric features were calculated as relative to the 2 target atoms.
\n\nType 2JHX\n- Found the atom on the shortest path between the 2 target atoms. So there were now 3 target atoms (atom index 0, atom index 1, atom on shortest path)\n- Connected each atom to the 3 target atoms on top of the default real bonds.\n- Features were calculated relative to all 3 target atoms e.g. distance & angle to atom index 0, atom index 1 and the atom on shortest path.
\n\nType 3JHX\n- Found the 2 atoms on the shortest path between the 2 target atoms. So there were now 4 target atoms (atom index 0, atom index 1, 1st atom on shortest path, 2nd atom on shortest path)\n- Connected each atom to the 4 target atoms on top of the default real bonds.\n- Features were calculated relative to all 4 target atoms.
\n\nAlso, I made all the graphs fully bidirectional. Using a fully bidirectional graph gave me a significant improvement over a one-directional graph which was used in the paper.
\n\nModel\n- The model was really basic with some additional layers and slightly larger dimensions, very similar to what is written here https://github.com/rusty1s/pytorch_geometric/blob/master/examples/qm9_nn_conv.py.\n- I added very little Dropout and BatchNorm in the initial linear transformation layer which actually led to the model performing better. \n- I experimented with adding Dropout in the MLP used by the NNConv and it showed promising results but they were too unstable so I decided to not go through with it.\n- I tried adding an attention mechanism over the messages passed by the network but did not see an improvement in score (most likely implemented it incorrectly)\n- I also tried using the node vectors of the target atoms only to predict the scc but this actually performed way worse (probably because the way I am representing my molecules does not translate well to using just the node vectors of a subset of nodes)\n- I only trained a single model for each type (8 models total) so did not do any ensembling
\n\nTrain only data\nUnfortunately, towards the end of the competition I was busy with some other work so could not get a chance to play around the fc, pso etc features.
", "rawMarkdown": "First of all, I want to thank everyone who participated in this competition and the competition organizers. \n\nThis was only my 2nd competition and I am really happy to have achieved my first ever medal and my first ever silver medal! \n\n**Overview**\n- Every coupling was treated as its own graph\n- For the same molecule, graphs of 2 different couplings were different from each other.\n- Used the MPNN from the Gilmer paper https://arxiv.org/abs/1704.01212\n- Used basic chemical features like atomic number and basic geometric features like angles and distances.\n- Had same features for all types but different connectivity for 1JHX, 2JHX and 3JHX\n- Most important part was not the model but how the molecular graph was connected together \n- All geometric features were relative to the atoms at atom index 0 and 1 and 1 or 2 other atoms which I found.\n\n**Molecular Graph Representation**\nIn the Gilmer Paper, a molecule is represented as a fully connected graph i.e. there are the default bonds (real bonds) and on top of that each atom is connected to each atom through a fake bond. In the paper, the point is to predict properties that belong to the whole graph and not to a particular edge or a node. So, in order to adapt to the nature of this competition, I used the following representation:\n\n- Each coupling was a data point i.e. each coupling was its own molecular graph\n- If a molecule had N number of couplings, then all N graphs are different from each other\n\n*Type 1JHX*\n- Connected each atom to the 2 target atoms (atom index 0 and 1) on top of the default real bonds (note how this is not the same as the Gilmer paper where the graph is fully connected)\n- All geometric features were calculated as relative to the 2 target atoms.\n\n*Type 2JHX*\n- Found the atom on the shortest path between the 2 target atoms. So there were now 3 target atoms (atom index 0, atom index 1, atom on shortest path)\n- Connected each atom to the 3 target atoms on top of the default real bonds.\n- Features were calculated relative to all 3 target atoms e.g. distance & angle to atom index 0, atom index 1 and the atom on shortest path.\n\n*Type 3JHX*\n- Found the 2 atoms on the shortest path between the 2 target atoms. So there were now 4 target atoms (atom index 0, atom index 1, 1st atom on shortest path, 2nd atom on shortest path)\n- Connected each atom to the 4 target atoms on top of the default real bonds.\n- Features were calculated relative to all 4 target atoms.\n\nAlso, I made all the graphs fully bidirectional. Using a fully bidirectional graph gave me a significant improvement over a one-directional graph which was used in the paper.\n\n**Model**\n- The model was really basic with some additional layers and slightly larger dimensions, very similar to what is written here https://github.com/rusty1s/pytorch_geometric/blob/master/examples/qm9_nn_conv.py.\n- I added very little Dropout and BatchNorm in the initial linear transformation layer which actually led to the model performing better. \n- I experimented with adding Dropout in the MLP used by the NNConv and it showed promising results but they were too unstable so I decided to not go through with it.\n- I tried adding an attention mechanism over the messages passed by the network but did not see an improvement in score (most likely implemented it incorrectly)\n- I also tried using the node vectors of the target atoms only to predict the scc but this actually performed way worse (probably because the way I am representing my molecules does not translate well to using just the node vectors of a subset of nodes)\n- I only trained a single model for each type (8 models total) so did not do any ensembling\n\n**Train only data**\nUnfortunately, towards the end of the competition I was busy with some other work so could not get a chance to play around the fc, pso etc features.", "votes": null }, { "id": "614047", "postDate": "08/31/2019 03:08:00", "content": "Congratulations on your medal and strong finish! That is an amazing start for your 2nd competition. Your write-up provides some nice insight into your approach.
", "rawMarkdown": "Congratulations on your medal and strong finish! That is an amazing start for your 2nd competition. Your write-up provides some nice insight into your approach.", "votes": null }, { "id": "615475", "postDate": "09/02/2019 02:08:39", "content": "Thanks for sharing !!
", "rawMarkdown": "Thanks for sharing !!", "votes": null } ], "comments": [ { "id": 614047, "author_name": "wjholst", "author_url": "", "post_date": "08/31/2019 03:08:00", "content": "Congratulations on your medal and strong finish! That is an amazing start for your 2nd competition. Your write-up provides some nice insight into your approach.
", "votes": null, "replies": [] }, { "id": 615475, "author_name": "silence2", "author_url": "", "post_date": "09/02/2019 02:08:39", "content": "Thanks for sharing !!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "613685": "First of all, I want to thank everyone who participated in this competition and the competition organizers. \n\nThis was only my 2nd competition and I am really happy to have achieved my first ever medal and my first ever silver medal! \n\n**Overview**\n- Every coupling was treated as its own graph\n- For the same molecule, graphs of 2 different couplings were different from each other.\n- Used the MPNN from the Gilmer paper https://arxiv.org/abs/1704.01212\n- Used basic chemical features like atomic number and basic geometric features like angles and distances.\n- Had same features for all types but different connectivity for 1JHX, 2JHX and 3JHX\n- Most important part was not the model but how the molecular graph was connected together \n- All geometric features were relative to the atoms at atom index 0 and 1 and 1 or 2 other atoms which I found.\n\n**Molecular Graph Representation**\nIn the Gilmer Paper, a molecule is represented as a fully connected graph i.e. there are the default bonds (real bonds) and on top of that each atom is connected to each atom through a fake bond. In the paper, the point is to predict properties that belong to the whole graph and not to a particular edge or a node. So, in order to adapt to the nature of this competition, I used the following representation:\n\n- Each coupling was a data point i.e. each coupling was its own molecular graph\n- If a molecule had N number of couplings, then all N graphs are different from each other\n\n*Type 1JHX*\n- Connected each atom to the 2 target atoms (atom index 0 and 1) on top of the default real bonds (note how this is not the same as the Gilmer paper where the graph is fully connected)\n- All geometric features were calculated as relative to the 2 target atoms.\n\n*Type 2JHX*\n- Found the atom on the shortest path between the 2 target atoms. So there were now 3 target atoms (atom index 0, atom index 1, atom on shortest path)\n- Connected each atom to the 3 target atoms on top of the default real bonds.\n- Features were calculated relative to all 3 target atoms e.g. distance & angle to atom index 0, atom index 1 and the atom on shortest path.\n\n*Type 3JHX*\n- Found the 2 atoms on the shortest path between the 2 target atoms. So there were now 4 target atoms (atom index 0, atom index 1, 1st atom on shortest path, 2nd atom on shortest path)\n- Connected each atom to the 4 target atoms on top of the default real bonds.\n- Features were calculated relative to all 4 target atoms.\n\nAlso, I made all the graphs fully bidirectional. Using a fully bidirectional graph gave me a significant improvement over a one-directional graph which was used in the paper.\n\n**Model**\n- The model was really basic with some additional layers and slightly larger dimensions, very similar to what is written here https://github.com/rusty1s/pytorch_geometric/blob/master/examples/qm9_nn_conv.py.\n- I added very little Dropout and BatchNorm in the initial linear transformation layer which actually led to the model performing better. \n- I experimented with adding Dropout in the MLP used by the NNConv and it showed promising results but they were too unstable so I decided to not go through with it.\n- I tried adding an attention mechanism over the messages passed by the network but did not see an improvement in score (most likely implemented it incorrectly)\n- I also tried using the node vectors of the target atoms only to predict the scc but this actually performed way worse (probably because the way I am representing my molecules does not translate well to using just the node vectors of a subset of nodes)\n- I only trained a single model for each type (8 models total) so did not do any ensembling\n\n**Train only data**\nUnfortunately, towards the end of the competition I was busy with some other work so could not get a chance to play around the fc, pso etc features.", "614047": "Congratulations on your medal and strong finish! That is an amazing start for your 2nd competition. Your write-up provides some nice insight into your approach.", "615475": "Thanks for sharing !!" }, "source": "existing_discussion", "complete": true, "competition_id": 14313, "competition_slug": "champs-scalar-coupling", "team_id": 3338759, "solution_url": "/competitions/champs-scalar-coupling/writeups/alamjeet-singh-35th-place-solution-basic-mpnn-firs" }