Hi All,
\n\nSo my first thought when starting this challenge was the Gilmer/Brain paper
\n\nIt uses the QM9 dataset for the prediction of bulk properties, so overall a somewhat similar task to what we have here, with the major difference that we are trying to predict edge properties.
\n\nFor fun I implemented it in this kernel here
\n\nI guess it is fairly similar to @hengck23 's implementation, although I haven't had a proper look at his implementation. And this one runs in a kernel.
\n\nI would just like to spur discussion. For starters, I find that if I try to use batch normalization the network really struggles. Why?
\n\nAnd maybe the output step is a bit wasteful as it makes a lot of predictions which are not used.
\n\nMaybe @jaechang can enlighten us as I think he is a subject matter expert
\n\n(as a note, this is not what I am using for my current score (2.085))
", "messages": [ { "id": "588260", "postDate": "07/30/2019 10:44:25", "content": "Hi All,
\n\nSo my first thought when starting this challenge was the Gilmer/Brain paper
\n\nIt uses the QM9 dataset for the prediction of bulk properties, so overall a somewhat similar task to what we have here, with the major difference that we are trying to predict edge properties.
\n\nFor fun I implemented it in this kernel here
\n\nI guess it is fairly similar to @hengck23 's implementation, although I haven't had a proper look at his implementation. And this one runs in a kernel.
\n\nI would just like to spur discussion. For starters, I find that if I try to use batch normalization the network really struggles. Why?
\n\nAnd maybe the output step is a bit wasteful as it makes a lot of predictions which are not used.
\n\nMaybe @jaechang can enlighten us as I think he is a subject matter expert
\n\n(as a note, this is not what I am using for my current score (2.085))
", "rawMarkdown": "Hi All,\n\nSo my first thought when starting this challenge was the [Gilmer/Brain paper](https://arxiv.org/abs/1704.01212)\n\nIt uses the QM9 dataset for the prediction of bulk properties, so overall a somewhat similar task to what we have here, with the major difference that we are trying to predict edge properties. \n \n\nFor fun I implemented it in this [kernel here](https://www.kaggle.com/fnands/1-mpnn)\n\nI guess it is fairly similar to @hengck23 's [implementation](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93972), although I haven't had a proper look at his implementation. And this one runs in a kernel. \n\nI would just like to spur discussion. For starters, I find that if I try to use batch normalization the network really struggles. Why? \n\nAnd maybe the output step is a bit wasteful as it makes a lot of predictions which are not used. \n\nMaybe @jaechang can enlighten us as I think he is a subject matter expert\n\n\n(as a note, this is not what I am using for my current score (2.085))", "votes": null }, { "id": "588747", "postDate": "07/31/2019 01:32:32", "content": "I hope that this information will help you!
", "rawMarkdown": "1. In this paper, though the authors didn't use MPNN, they used batch normalization in graph neural network(https://link.springer.com/article/10.1186/s13321-018-0287-6)\n2. I am using a variant of MPNN. I achieved about -2.1 in the leaderboard without ensemble using basic MPNN model. \n3. After the propagation step in MPNN, my model predicts label using states vectors of two nodes and use only pairs having labels. \n\nI hope that this information will help you!", "votes": null }, { "id": "588827", "postDate": "07/31/2019 05:24:25", "content": "I was using a single tree based model. I was hoping to get a validation that MPNN works here before I would invest time there. @jaechang Thanks for disclosing that !!
", "rawMarkdown": "I was using a single tree based model. I was hoping to get a validation that MPNN works here before I would invest time there. @jaechang Thanks for disclosing that !!", "votes": null }, { "id": "588867", "postDate": "07/31/2019 06:59:49", "content": "Seems it is time to look into MPNN ;)
", "rawMarkdown": "Seems it is time to look into MPNN ;)", "votes": null }, { "id": "589019", "postDate": "07/31/2019 10:50:41", "content": "@jaechang , thanks a lot! Will read paper.
", "rawMarkdown": "jaechang , thanks a lot! Will read paper.", "votes": null }, { "id": "589453", "postDate": "07/31/2019 23:06:02", "content": "Lim, are you training your neural network individually for each type or are you predicting the coupling constant for all types in one model?
", "rawMarkdown": "Lim, are you training your neural network individually for each type or are you predicting the coupling constant for all types in one model?", "votes": null }, { "id": "589477", "postDate": "08/01/2019 00:12:49", "content": "Thanks for your sharing. Could you elaborate more on point 3 you shared?
", "rawMarkdown": "Thanks for your sharing. Could you elaborate more on point 3 you shared?", "votes": null }, { "id": "589581", "postDate": "08/01/2019 05:57:07", "content": "thank you！ by a way, \"from the two node vectors, their coupling constant is predicted\", do you use global pooling of a module?
", "rawMarkdown": "thank you！ by a way, \"from the two node vectors, their coupling constant is predicted\", do you use global pooling of a module?", "votes": null } ], "comments": [ { "id": 588747, "author_name": "jaechang", "author_url": "", "post_date": "07/31/2019 01:32:32", "content": "I hope that this information will help you!
", "votes": null, "replies": [ { "id": 588867, "author_name": "cpmpml", "author_url": "", "post_date": "07/31/2019 06:59:49", "content": "Seems it is time to look into MPNN ;)
", "votes": null, "replies": [] }, { "id": 589019, "author_name": "fnands", "author_url": "", "post_date": "07/31/2019 10:50:41", "content": "@jaechang , thanks a lot! Will read paper.
", "votes": null, "replies": [] }, { "id": 589453, "author_name": "petersk20", "author_url": "", "post_date": "07/31/2019 23:06:02", "content": "Lim, are you training your neural network individually for each type or are you predicting the coupling constant for all types in one model?
", "votes": null, "replies": [] }, { "id": 589477, "author_name": "xiaokangwang", "author_url": "", "post_date": "08/01/2019 00:12:49", "content": "Thanks for your sharing. Could you elaborate more on point 3 you shared?
", "votes": null, "replies": [] }, { "id": 589581, "author_name": "jaechang", "author_url": "", "post_date": "08/01/2019 05:57:07", "content": "thank you！ by a way, \"from the two node vectors, their coupling constant is predicted\", do you use global pooling of a module?
", "votes": null, "replies": [] } ] }, { "id": 588827, "author_name": "silence2", "author_url": "", "post_date": "07/31/2019 05:24:25", "content": "I was using a single tree based model. I was hoping to get a validation that MPNN works here before I would invest time there. @jaechang Thanks for disclosing that !!
", "votes": null, "replies": [] } ], "raw_markdown_by_id": { "588260": "Hi All,\n\nSo my first thought when starting this challenge was the [Gilmer/Brain paper](https://arxiv.org/abs/1704.01212)\n\nIt uses the QM9 dataset for the prediction of bulk properties, so overall a somewhat similar task to what we have here, with the major difference that we are trying to predict edge properties. \n \n\nFor fun I implemented it in this [kernel here](https://www.kaggle.com/fnands/1-mpnn)\n\nI guess it is fairly similar to @hengck23 's [implementation](https://www.kaggle.com/c/champs-scalar-coupling/discussion/93972), although I haven't had a proper look at his implementation. And this one runs in a kernel. \n\nI would just like to spur discussion. For starters, I find that if I try to use batch normalization the network really struggles. Why? \n\nAnd maybe the output step is a bit wasteful as it makes a lot of predictions which are not used. \n\nMaybe @jaechang can enlighten us as I think he is a subject matter expert\n\n\n(as a note, this is not what I am using for my current score (2.085))", "588747": "1. In this paper, though the authors didn't use MPNN, they used batch normalization in graph neural network(https://link.springer.com/article/10.1186/s13321-018-0287-6)\n2. I am using a variant of MPNN. I achieved about -2.1 in the leaderboard without ensemble using basic MPNN model. \n3. After the propagation step in MPNN, my model predicts label using states vectors of two nodes and use only pairs having labels. \n\nI hope that this information will help you!", "588827": "I was using a single tree based model. I was hoping to get a validation that MPNN works here before I would invest time there. @jaechang Thanks for disclosing that !!", "588867": "Seems it is time to look into MPNN ;)", "589019": "jaechang , thanks a lot! Will read paper.", "589453": "Lim, are you training your neural network individually for each type or are you predicting the coupling constant for all types in one model?", "589477": "Thanks for your sharing. Could you elaborate more on point 3 you shared?", "589581": "1. @petersk20 In my model, the propagating module is shared for all types and there are 8 final fully connected layer s which correspond to each type.\n3. @xiaokangwang After the propagation step, each node (=atom) has its own node vectors. Then, from the two node vectors, their coupling constant is predicted.", "589584": "thank you！ by a way, \"from the two node vectors, their coupling constant is predicted\", do you use global pooling of a module?" }, "source": "meta" }